-- Logs begin at Fri 2022-05-06 14:03:42 UTC, end at Fri 2022-05-06 15:20:00 UTC. -- May 06 14:43:22.487143 dv-cinder-840678-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 06 14:43:22.495845 dv-cinder-840678-2-iscsi devstack@keystone.service[79879]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 06 14:43:22.497874 dv-cinder-840678-2-iscsi devstack@keystone.service[79879]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri May 6 14:43:22 2022] *** May 06 14:43:22.497874 dv-cinder-840678-2-iscsi devstack@keystone.service[79879]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 06 14:43:22.497874 dv-cinder-840678-2-iscsi devstack@keystone.service[79879]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 06 14:43:22.497874 dv-cinder-840678-2-iscsi devstack@keystone.service[79879]: nodename: dv-cinder-840678-2-iscsi May 06 14:43:22.497874 dv-cinder-840678-2-iscsi devstack@keystone.service[79879]: machine: x86_64 May 06 14:43:22.497874 dv-cinder-840678-2-iscsi devstack@keystone.service[79879]: clock source: unix May 06 14:43:22.497874 dv-cinder-840678-2-iscsi devstack@keystone.service[79879]: pcre jit disabled May 06 14:43:22.497874 dv-cinder-840678-2-iscsi devstack@keystone.service[79879]: detected number of CPU cores: 4 May 06 14:43:22.497874 dv-cinder-840678-2-iscsi devstack@keystone.service[79879]: current working directory: / May 06 14:43:22.497874 dv-cinder-840678-2-iscsi devstack@keystone.service[79879]: detected binary path: /usr/bin/uwsgi-core May 06 14:43:22.497874 dv-cinder-840678-2-iscsi devstack@keystone.service[79879]: your processes number limit is 47680 May 06 14:43:22.498449 dv-cinder-840678-2-iscsi devstack@keystone.service[79879]: your memory page size is 4096 bytes May 06 14:43:22.498449 dv-cinder-840678-2-iscsi devstack@keystone.service[79879]: detected max file descriptor number: 2048 May 06 14:43:22.498449 dv-cinder-840678-2-iscsi devstack@keystone.service[79879]: lock engine: pthread robust mutexes May 06 14:43:22.498449 dv-cinder-840678-2-iscsi devstack@keystone.service[79879]: thunder lock: enabled May 06 14:43:22.498449 dv-cinder-840678-2-iscsi devstack@keystone.service[79879]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 06 14:43:22.498449 dv-cinder-840678-2-iscsi devstack@keystone.service[79879]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 06 14:43:22.544777 dv-cinder-840678-2-iscsi devstack@keystone.service[79879]: Python main interpreter initialized at 0x5623d70779c0 May 06 14:43:22.544777 dv-cinder-840678-2-iscsi devstack@keystone.service[79879]: python threads support enabled May 06 14:43:22.544777 dv-cinder-840678-2-iscsi devstack@keystone.service[79879]: your server socket listen backlog is limited to 100 connections May 06 14:43:22.544777 dv-cinder-840678-2-iscsi devstack@keystone.service[79879]: your mercy for graceful operations on workers is 90 seconds May 06 14:43:22.545170 dv-cinder-840678-2-iscsi devstack@keystone.service[79879]: mapped 403077 bytes (393 KB) for 2 cores May 06 14:43:22.545245 dv-cinder-840678-2-iscsi devstack@keystone.service[79879]: *** Operational MODE: preforking *** May 06 14:43:22.545494 dv-cinder-840678-2-iscsi devstack@keystone.service[79879]: *** uWSGI is running in multiple interpreter mode *** May 06 14:43:22.545494 dv-cinder-840678-2-iscsi devstack@keystone.service[79879]: spawned uWSGI master process (pid: 79879) May 06 14:43:22.545708 dv-cinder-840678-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 06 14:43:22.547648 dv-cinder-840678-2-iscsi devstack@keystone.service[79879]: spawned uWSGI worker 1 (pid: 79887, cores: 1) May 06 14:43:22.547648 dv-cinder-840678-2-iscsi devstack@keystone.service[79879]: spawned uWSGI worker 2 (pid: 79888, cores: 1) May 06 14:43:22.547648 dv-cinder-840678-2-iscsi devstack@keystone.service[79879]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 06 14:43:28.335493 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 06 14:43:28.335493 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: from cryptography.utils import int_from_bytes May 06 14:43:28.336250 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 06 14:43:28.336250 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: from cryptography.utils import int_from_bytes May 06 14:43:28.336890 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 06 14:43:28.336890 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: from cryptography.utils import int_from_bytes May 06 14:43:28.337334 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 06 14:43:28.337334 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: from cryptography.utils import int_from_bytes May 06 14:43:28.386693 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 06 14:43:28.386886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 06 14:43:28.387084 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] command line args: [] {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 06 14:43:28.387323 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 06 14:43:28.387491 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ================================================================================ {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 06 14:43:28.387748 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] admin_token = **** {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.387991 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] config_dir = [] {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.388178 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.388360 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] config_source = [] {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.388537 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] debug = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.388761 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.391261 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.391261 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] insecure_debug = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.391261 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.391261 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.391261 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] list_limit = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.391261 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] log_config_append = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.391261 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.391261 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] log_dir = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.391261 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] log_file = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.391741 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 06 14:43:28.391741 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 06 14:43:28.391741 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] command line args: [] {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 06 14:43:28.391901 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.391901 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.392004 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 06 14:43:28.392004 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ================================================================================ {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 06 14:43:28.392114 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.392176 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] admin_token = **** {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.392237 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.392237 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.392352 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] config_dir = [] {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.392418 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.392482 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.392482 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] config_source = [] {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.392583 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.393219 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] debug = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.393407 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.393603 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.393855 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.394024 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.394221 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.394410 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] insecure_debug = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.394606 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.394796 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.394983 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.395214 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.395436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.395605 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] list_limit = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.395605 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] log_config_append = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.396042 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] notification_format = cadf {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.396207 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.396394 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.396589 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] log_dir = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.396589 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] log_file = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.396589 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.397168 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.397168 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] publish_errors = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.397532 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.397726 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.397947 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.398087 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.398257 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.398391 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.398517 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.398696 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.398834 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.398970 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.399098 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.399230 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] standard_threads = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.399399 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.399537 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] strict_password_check = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.399678 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.399819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.399947 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.400072 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] use_eventlog = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.400759 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.401361 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] use_journal = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.401361 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] use_json = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.401606 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.401731 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] use_stderr = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.401920 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.402062 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] use_syslog = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.402196 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] notification_format = cadf {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.402323 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] watch_log_file = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.402323 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.402578 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.402717 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.402717 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.402960 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.402960 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] publish_errors = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.403216 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.403409 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.403547 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.403680 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.403806 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.403931 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.404081 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.404214 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.404357 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.404498 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.404651 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.404804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] standard_threads = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.404942 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.405088 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] strict_password_check = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.405220 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.405393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.405393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] use_eventlog = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.405646 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.406045 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] use_journal = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.406177 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.406304 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] use_json = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.406304 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] use_stderr = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.406550 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.406682 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] use_syslog = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.406883 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.407017 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] watch_log_file = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 06 14:43:28.407144 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.407293 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.407432 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.407581 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.407718 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.407847 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.407973 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.408099 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.408233 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.408379 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.408513 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.408667 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.408805 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.408933 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.409058 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.409194 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.409327 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.409461 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.409593 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.409720 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.409845 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.409979 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.410106 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.410286 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.410430 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.410557 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.410694 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.410831 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.410958 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.411098 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.411224 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.411349 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.411489 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.411623 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.411756 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.411895 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.412032 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.412170 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.412300 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.412434 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.412575 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.412761 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.412902 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.413039 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.413178 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.413381 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.413521 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.413651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.413782 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.413782 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.414031 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.414165 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.414291 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.414417 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.414547 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.414680 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.414806 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.414940 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.415067 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.415193 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.415324 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.415450 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.415583 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.415720 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.416558 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.417171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.417324 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.417455 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.417594 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.417722 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.417849 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.417981 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.418111 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.418237 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.418371 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.418498 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.419749 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.419920 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.420112 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.420307 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.420531 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.420751 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.420914 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.421102 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.421288 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.421475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.421703 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.421870 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.422128 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.422325 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.422504 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.422694 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.422915 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.423121 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.423319 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.423519 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.423683 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.423873 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.424098 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.424269 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.424466 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.424729 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.424895 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.425039 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.425211 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.425211 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.425495 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.425636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.425765 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.425931 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.426069 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.426231 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.426380 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.426509 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.426635 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.426761 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.426887 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.427030 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.427159 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.427159 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.enabled = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.427159 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.427159 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.427159 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.427801 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.427930 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.428056 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.428187 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.428315 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.428441 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.428575 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.428742 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.428882 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.429011 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.429143 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.429272 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.429407 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.429542 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.429676 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.429803 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.429929 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.430055 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.430215 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.430446 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.430584 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.430722 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.430849 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.430975 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.431101 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.431235 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.431395 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.431595 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.432132 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.432268 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.432396 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.432534 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.432734 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.enabled = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.432864 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.433011 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.433139 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.433265 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.433399 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.433529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.433654 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.433780 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.433906 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.434041 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.434178 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.434317 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.434445 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.434573 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.434704 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.434847 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.434981 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.434981 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] auth.external = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.435231 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.435358 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] auth.mapped = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.435483 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.435612 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.435740 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.435874 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.436001 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.436127 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] auth.password = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.436253 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.436378 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] auth.token = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.436504 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.436668 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.436802 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.436929 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] catalog.caching = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.437061 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.437189 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.437315 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.437450 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.437577 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.437704 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.437840 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.437974 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.438102 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.438236 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.438400 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.438543 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] credential.caching = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.438672 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.438800 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] credential.driver = sql {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.438935 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.439063 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.439191 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.439326 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.439454 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.439581 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.439708 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.439836 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.439962 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.440097 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.440235 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.440362 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.440489 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] auth.external = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.440648 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.440783 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] auth.mapped = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.440926 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.441056 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.441190 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.441319 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.441445 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.441581 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] auth.password = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.441709 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.441836 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] auth.token = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.441974 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.442110 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.442110 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] catalog.caching = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.442360 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.442489 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.442617 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.442744 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.442878 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] federation.caching = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.443007 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.443143 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.443272 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.443400 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] federation.driver = sql {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.443526 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.443653 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.443790 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] credential.caching = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.443925 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.444054 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] credential.driver = sql {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.444182 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.444310 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.444436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.444436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.444712 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.444712 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.444956 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.445084 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.445211 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.445343 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.445492 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.445622 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.445749 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.445877 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.446004 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] identity.caching = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.446144 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.446349 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.446479 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.446607 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.446607 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.446857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.447011 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.447305 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.447810 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] identity.driver = sql {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.447964 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.448098 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.448227 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.448227 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.448227 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] federation.caching = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.448593 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.448757 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.448891 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.449018 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] federation.driver = sql {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.449145 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.449285 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.449414 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.449541 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.449668 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.449805 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.449940 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.450076 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.450213 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.450356 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.450489 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.450617 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.450758 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.450895 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.451023 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.451152 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.451279 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.451408 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.451544 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.451681 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] identity.caching = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.451809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.451935 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.452062 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.452189 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.452324 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.452453 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.452580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.452741 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.452871 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.452997 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] identity.driver = sql {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.453132 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.453272 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.453415 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.453543 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.453683 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.453818 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.453954 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.453954 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.454195 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.454322 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.454450 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.454590 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.454728 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.454856 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.454856 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.455120 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.455248 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.455648 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.455789 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.455917 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.456044 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.456171 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.456309 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.password = **** {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.456440 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.456577 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.456737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.456866 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.457006 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.457137 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.457275 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.457428 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.457558 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.457705 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.457837 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.457965 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.458092 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.458230 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.458360 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.458496 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.458624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.458752 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.458879 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.459028 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.459158 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.459286 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.459286 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.459535 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.459535 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.459783 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.459783 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.460025 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.460159 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.user = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.463620 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.466421 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.466595 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.466745 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.466903 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.467047 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.467184 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.467327 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.467473 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.467473 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.467729 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.467863 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.password = **** {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.467996 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.468131 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.468277 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.468277 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.468532 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.468690 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.468837 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.468972 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.469117 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.469255 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.469388 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.469521 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.469654 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.469787 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.469929 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.470064 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.470199 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.470332 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.470464 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.470608 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.470750 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.470885 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.471188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.471359 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.471504 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.471639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.471781 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.471916 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.472050 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.472183 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.472318 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.user = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.472451 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.472592 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.472779 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.472779 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.473068 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.473068 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.473411 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.473411 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.473690 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.473832 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] policy.driver = sql {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.473972 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.474111 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.474259 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.474402 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.474541 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.474679 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.474828 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.474967 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] receipt.caching = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.475114 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.475254 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.475393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.475393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.475656 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.475794 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.475941 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.475941 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.476204 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.476341 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.476480 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.476653 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] resource.caching = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.476812 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.476953 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.477091 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.477228 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] resource.driver = sql {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.477366 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.477504 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.477650 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.477789 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.478186 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.479004 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.479147 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.479285 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] revoke.caching = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.479433 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.479585 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.479725 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.479933 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.480078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.480224 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] role.cache_time = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.480439 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.480581 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] role.caching = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.480747 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.480886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] role.driver = sql {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.481024 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.481162 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] role.list_limit = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.481313 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] policy.driver = sql {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.481453 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.481592 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.481592 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.481865 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.482004 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.482150 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.482289 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] receipt.caching = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.482426 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.482564 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.482701 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.482840 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.482986 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.482986 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.483248 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.483422 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.483422 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.483700 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.483860 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.484005 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.484143 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.484281 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] resource.caching = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.484437 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.484732 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.484924 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.485069 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] resource.driver = sql {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.485069 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.485331 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.485331 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.485592 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.485592 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.485876 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.486017 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] revoke.caching = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.486162 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.486302 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.486439 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.486586 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.486731 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.486870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] role.cache_time = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.487015 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.487156 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] role.caching = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.487302 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.487448 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] role.driver = sql {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.487598 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.487737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] role.list_limit = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.487874 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.488013 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.488158 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.488299 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.488438 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.488575 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.488737 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.488877 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.489027 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.489168 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.489307 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.489455 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.489593 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.489731 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.489877 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.490016 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.490216 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.490361 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.490500 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] token.cache_time = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.490500 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] token.caching = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.490778 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.490918 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.491132 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.491290 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] token.provider = fernet {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.491430 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.491579 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.491727 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.491867 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.492005 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.492144 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.492282 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.492419 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.492566 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.492735 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.492876 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.493015 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.493152 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.493291 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] trust.driver = sql {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.493452 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.493594 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.493732 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.494063 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.494611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.494862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.495016 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.495157 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.495295 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.495435 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.495573 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.495712 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.495930 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.496134 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.496343 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.496497 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.496767 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.496956 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] database.connection = **** {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.497234 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.497444 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.497632 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.497827 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.498021 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.498163 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.498302 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.498449 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.498589 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] token.cache_time = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.498738 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.498877 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] token.caching = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.498877 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.498877 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] token.provider = fernet {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.498877 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.498877 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.499517 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.499657 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.499795 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.499933 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.500079 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.500219 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.500356 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.500494 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.500699 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.500892 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] trust.driver = sql {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.501105 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.501294 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.501489 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.501689 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.501832 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.501972 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.502119 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.502260 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.502398 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.502537 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.502684 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.502825 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.502982 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.503122 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.503260 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.503260 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=79887) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 06 14:43:28.503523 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.503523 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] database.connection = **** {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.503523 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.503523 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.503523 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.503523 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.503523 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.503523 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.503523 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.504960 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.504960 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.504960 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.504960 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.504960 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.504960 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.504960 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.504960 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.504960 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.506341 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.506341 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 06 14:43:28.506341 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=79888) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 06 14:43:28.506927 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.507160 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.507382 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.508153 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.508393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.508700 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.516797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.518115 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.546115 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.546357 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.547327 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.547562 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.600827 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.606180 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.653077 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.653354 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.656470 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.656684 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.720449 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.720750 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.720903 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.721130 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.721297 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.721448 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.721592 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.721732 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.721882 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.721882 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.722149 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.722288 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.744989 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.745228 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.745473 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.745682 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.745827 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.745969 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.746108 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.746108 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.746388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.746527 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.746666 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.746666 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.746939 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.746939 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.757621 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.764059 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.772998 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.773216 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.773444 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.773669 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.773890 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.774105 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.774322 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=79888) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.778121 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.778358 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.778556 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.778777 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.778990 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.779198 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.779406 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=79887) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 06 14:43:28.793591 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.793819 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 06 14:43:28.794127 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.794329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 06 14:43:28.794599 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.794860 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.795118 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.795373 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.799266 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.799470 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 06 14:43:28.799828 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.800042 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 06 14:43:28.800321 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.800584 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.800857 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.801126 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.807395 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.807671 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.807937 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.808213 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.813424 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.813698 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.813964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.814240 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.816395 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=79888) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 06 14:43:28.819729 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=79888) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 06 14:43:28.820011 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.820293 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.820711 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.821001 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.821264 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.821537 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.821810 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.822175 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.822453 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.822727 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.822893 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=79887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 06 14:43:28.826155 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=79887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 06 14:43:28.826431 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.826708 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.827093 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.827372 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.827627 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.827894 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.828163 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.828428 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.828722 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.828986 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.849971 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.851586 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=79888) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 06 14:43:28.851864 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.857672 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.857872 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=79888) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 06 14:43:28.858058 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.858339 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.859285 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=79887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 06 14:43:28.859559 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.864802 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=79887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 06 14:43:28.865077 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.865350 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.865656 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=79888) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 06 14:43:28.865936 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.870649 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=79888) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 06 14:43:28.870934 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.871216 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.871498 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.871773 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.872045 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.872314 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.872582 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.873042 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=79887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 06 14:43:28.873319 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.877894 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=79887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 06 14:43:28.878178 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.878455 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.878727 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.878993 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.879257 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.879520 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.879779 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.891470 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.891754 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.896351 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=79888) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 06 14:43:28.900041 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.900903 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.904291 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.904571 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.906206 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=79888) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 06 14:43:28.909397 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=79887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 06 14:43:28.909872 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=79888) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 06 14:43:28.912966 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.913253 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.913658 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.913957 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.914242 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.914524 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.914801 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.915087 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.919346 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=79887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 06 14:43:28.927212 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=79887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 06 14:43:28.931149 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.931580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.931964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.932353 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.932762 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.933145 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.934983 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=79888) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 06 14:43:28.935361 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.935631 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.935896 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.943820 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.945565 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.945837 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.949138 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=79888) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 06 14:43:28.949419 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.949696 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.949982 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.950261 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.953810 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=79887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 06 14:43:28.954295 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.954667 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.955053 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.963631 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.964273 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=79888) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 06 14:43:28.964554 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.964861 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.965150 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.965426 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.965695 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.965974 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.966182 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.966556 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.969863 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=79887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 06 14:43:28.970278 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.970664 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.971058 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.971463 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.985241 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=79888) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 06 14:43:28.986237 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=79887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 06 14:43:28.986637 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.987025 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.987427 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.987818 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.988262 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.988580 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=79888) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 06 14:43:28.988710 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.991919 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.993539 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.995904 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=79888) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 06 14:43:28.996234 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:28.996470 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.004110 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.005752 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=79888) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 06 14:43:29.008330 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=79887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 06 14:43:29.009022 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.009312 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.010195 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.010472 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.012808 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=79887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 06 14:43:29.016238 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.017993 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.019739 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=79887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 06 14:43:29.019827 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=79888) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 06 14:43:29.020095 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.020274 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.020383 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.020768 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.028836 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.028962 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=79888) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 06 14:43:29.029242 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.029523 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.029808 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.030082 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.030363 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.030652 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.031199 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.031468 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.031651 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=79887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 06 14:43:29.031743 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.031990 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.032259 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.032536 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.032841 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=79888) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.035459 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.035857 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.036245 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.036664 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.046284 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=79887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 06 14:43:29.046699 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.047083 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.055160 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=79887) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 06 14:43:29.055549 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.055931 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.056308 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.057306 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.057699 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.058076 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.058495 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.058882 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.059284 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.059658 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.060039 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.060410 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.060800 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=79887) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 06 14:43:29.073072 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5623d70779c0 pid: 79888 (default app) May 06 14:43:29.076460 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e2891a-b5c9-4f20-a352-fb32199ce71c None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:29.076791 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e2891a-b5c9-4f20-a352-fb32199ce71c None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:29.077076 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e2891a-b5c9-4f20-a352-fb32199ce71c None None] PATH_INFO: `/v3` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:29.082967 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1/1] 10.222.0.23 () {50 vars in 791 bytes} [Fri May 6 14:43:29 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:43:29.101119 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5623d70779c0 pid: 79887 (default app) May 06 14:43:37.173200 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6423505-c9d1-468f-a2fe-9bfaef6a7212 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:37.173738 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6423505-c9d1-468f-a2fe-9bfaef6a7212 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:37.173916 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6423505-c9d1-468f-a2fe-9bfaef6a7212 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:37.175345 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 2/2] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:43:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:43:37.183060 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a381e9cf-7238-4e95-830c-ef4e02e4810f None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:37.183467 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a381e9cf-7238-4e95-830c-ef4e02e4810f None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:37.183636 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a381e9cf-7238-4e95-830c-ef4e02e4810f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:37.214337 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG oslo_db.sqlalchemy.engines [None req-a381e9cf-7238-4e95-830c-ef4e02e4810f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=79887) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 06 14:43:37.401772 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG passlib.handlers.bcrypt [None req-a381e9cf-7238-4e95-830c-ef4e02e4810f None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=79887) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 06 14:43:37.402178 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG passlib.handlers.bcrypt [None req-a381e9cf-7238-4e95-830c-ef4e02e4810f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=79887) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 06 14:43:37.457810 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-a381e9cf-7238-4e95-830c-ef4e02e4810f None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:43:37.459388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-a381e9cf-7238-4e95-830c-ef4e02e4810f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:43:37.586312 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1/3] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:43:37 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 06 14:43:37.753959 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6423505-c9d1-468f-a2fe-9bfaef6a7212 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:37.790182 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG oslo_db.sqlalchemy.engines [None req-f6423505-c9d1-468f-a2fe-9bfaef6a7212 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=79888) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 06 14:43:37.918124 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6423505-c9d1-468f-a2fe-9bfaef6a7212 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:37.922440 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e4ea58-8b17-43f4-ab40-2221b012d1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:37.923104 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4ea58-8b17-43f4-ab40-2221b012d1cc admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:37.923392 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4ea58-8b17-43f4-ab40-2221b012d1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:37.923672 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4ea58-8b17-43f4-ab40-2221b012d1cc admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:37.932996 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e4ea58-8b17-43f4-ab40-2221b012d1cc admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:37.987005 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b0e4ea58-8b17-43f4-ab40-2221b012d1cc admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 06 14:43:37.987345 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e4ea58-8b17-43f4-ab40-2221b012d1cc admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:37.998839 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 3/4] 10.222.0.23 () {58 vars in 1324 bytes} [Fri May 6 14:43:37 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 06 14:43:38.004699 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a381e9cf-7238-4e95-830c-ef4e02e4810f None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:38.019606 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a381e9cf-7238-4e95-830c-ef4e02e4810f None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:38.024043 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67987aa-5e69-4847-9241-863b08e098ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:38.025475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67987aa-5e69-4847-9241-863b08e098ba admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:38.025596 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67987aa-5e69-4847-9241-863b08e098ba admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:38.025824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67987aa-5e69-4847-9241-863b08e098ba admin admin] PATH_INFO: `/v3/projects/ccc1ae27e989468199e57e6588faed16` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:38.027346 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67987aa-5e69-4847-9241-863b08e098ba admin admin] RBAC: Authorizing `identity:get_project(project_id=ccc1ae27e989468199e57e6588faed16)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:38.079471 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a67987aa-5e69-4847-9241-863b08e098ba admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 06 14:43:38.079816 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67987aa-5e69-4847-9241-863b08e098ba admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:38.084503 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 2/5] 10.222.0.23 () {56 vars in 1207 bytes} [Fri May 6 14:43:38 2022] GET /identity/v3/projects/ccc1ae27e989468199e57e6588faed16 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:43:40.351372 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e17565-8353-4b06-99a6-2c3f0335ae3a None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:40.351900 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e17565-8353-4b06-99a6-2c3f0335ae3a None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:40.351900 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e17565-8353-4b06-99a6-2c3f0335ae3a None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:40.357253 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 4/6] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:43:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:43:40.362567 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71839e0-0d85-45be-9a9e-e5f96a402ea1 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:40.362975 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71839e0-0d85-45be-9a9e-e5f96a402ea1 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:40.363192 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71839e0-0d85-45be-9a9e-e5f96a402ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:40.383176 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-f71839e0-0d85-45be-9a9e-e5f96a402ea1 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:43:40.385032 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-f71839e0-0d85-45be-9a9e-e5f96a402ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:43:40.392906 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 3/7] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:43:40 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 06 14:43:40.416309 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e17565-8353-4b06-99a6-2c3f0335ae3a None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:40.428320 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e17565-8353-4b06-99a6-2c3f0335ae3a None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:40.432655 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51db1542-dfba-414c-b7ee-5cb85478d470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:40.433298 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db1542-dfba-414c-b7ee-5cb85478d470 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:40.433604 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db1542-dfba-414c-b7ee-5cb85478d470 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:40.433871 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db1542-dfba-414c-b7ee-5cb85478d470 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:40.441496 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51db1542-dfba-414c-b7ee-5cb85478d470 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:40.443267 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51db1542-dfba-414c-b7ee-5cb85478d470 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:40.452262 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 5/8] 10.222.0.23 () {58 vars in 1324 bytes} [Fri May 6 14:43:40 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 06 14:43:40.457756 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71839e0-0d85-45be-9a9e-e5f96a402ea1 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:40.467439 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71839e0-0d85-45be-9a9e-e5f96a402ea1 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:40.471847 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ffa0a-b782-40e4-9471-ef8be264a5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:40.472750 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ffa0a-b782-40e4-9471-ef8be264a5b2 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:40.473039 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ffa0a-b782-40e4-9471-ef8be264a5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:40.473319 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ffa0a-b782-40e4-9471-ef8be264a5b2 admin admin] PATH_INFO: `/v3/users/66f1687219494ee0a402084110e075fd` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:40.474760 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8ffa0a-b782-40e4-9471-ef8be264a5b2 admin admin] RBAC: Authorizing `identity:get_user(user_id=66f1687219494ee0a402084110e075fd)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:40.476583 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8ffa0a-b782-40e4-9471-ef8be264a5b2 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:40.478775 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 4/9] 10.222.0.23 () {56 vars in 1198 bytes} [Fri May 6 14:43:40 2022] GET /identity/v3/users/66f1687219494ee0a402084110e075fd => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:43:42.800538 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89c1e7-3f7a-4c1b-9ae0-a885bd36dd35 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:42.801101 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89c1e7-3f7a-4c1b-9ae0-a885bd36dd35 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:42.801101 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89c1e7-3f7a-4c1b-9ae0-a885bd36dd35 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:42.802611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 6/10] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:43:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:43:42.808411 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c840be-b2d4-423c-bf5b-aac7d577c898 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:42.808819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c840be-b2d4-423c-bf5b-aac7d577c898 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:42.809010 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c840be-b2d4-423c-bf5b-aac7d577c898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:42.861057 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73e6d6-26fd-465f-af75-894427d2cc5a None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:42.861351 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73e6d6-26fd-465f-af75-894427d2cc5a None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:42.861633 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73e6d6-26fd-465f-af75-894427d2cc5a None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:42.863276 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 7/11] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:43:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:43:42.869632 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c83470-54ed-4bb0-8c4c-cbcfbc4eebe0 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:42.869987 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c83470-54ed-4bb0-8c4c-cbcfbc4eebe0 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:42.870175 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c83470-54ed-4bb0-8c4c-cbcfbc4eebe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:42.911293 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG passlib.handlers.bcrypt [None req-b5c83470-54ed-4bb0-8c4c-cbcfbc4eebe0 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=79888) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 06 14:43:42.911725 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG passlib.handlers.bcrypt [None req-b5c83470-54ed-4bb0-8c4c-cbcfbc4eebe0 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=79888) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 06 14:43:42.934412 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-60c840be-b2d4-423c-bf5b-aac7d577c898 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:43:42.935620 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-60c840be-b2d4-423c-bf5b-aac7d577c898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:43:42.959242 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-b5c83470-54ed-4bb0-8c4c-cbcfbc4eebe0 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:43:42.960872 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-b5c83470-54ed-4bb0-8c4c-cbcfbc4eebe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:43:43.013311 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 5/12] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:43:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 06 14:43:43.036404 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c840be-b2d4-423c-bf5b-aac7d577c898 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:43.060148 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 8/13] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:43:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 06 14:43:43.081747 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c83470-54ed-4bb0-8c4c-cbcfbc4eebe0 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:43.132228 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c840be-b2d4-423c-bf5b-aac7d577c898 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:43.166724 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcaacbf-df3d-4f64-bcd3-d0436fb59851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:43.167524 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcaacbf-df3d-4f64-bcd3-d0436fb59851 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:43.167809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcaacbf-df3d-4f64-bcd3-d0436fb59851 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:43.168088 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcaacbf-df3d-4f64-bcd3-d0436fb59851 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:43.179321 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcaacbf-df3d-4f64-bcd3-d0436fb59851 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:43:43.180160 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c83470-54ed-4bb0-8c4c-cbcfbc4eebe0 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:43.190436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-0bcaacbf-df3d-4f64-bcd3-d0436fb59851 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:43:43.190436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:43:43.190436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:43:43.190436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:43:43.190436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:43:43.190436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:43:43.190436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:43:43.190436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:43:43.190436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:43:43.190436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:43:43.190436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:43:43.190436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:43:43.190436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:43:43.190436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:43:43.190436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:43:43.190436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:43:43.191428 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:43:43.191428 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:43:43.191428 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:43:43.191428 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:43:43.191428 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:43:43.191428 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:43:43.191428 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:43:43.191428 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:43:43.191428 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:43:43.191428 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:43:43.191428 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:43:43.191428 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:43:43.191428 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:43:43.191428 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:43:43.191428 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:43:43.191428 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:43:43.192391 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:43:43.192391 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:43:43.192391 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:43:43.192391 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:43:43.192391 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:43:43.192391 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:43:43.192391 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:43:43.192391 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:43:43.192391 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:43:43.192391 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:43:43.192391 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:43:43.192391 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:43:43.192391 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 6/14] 10.222.0.23 () {56 vars in 1096 bytes} [Fri May 6 14:43:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:43:43.197302 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcaacbf-df3d-4f64-bcd3-d0436fb59851 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:43.214798 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c4dd77-f9ec-467f-91cb-d4bdad51dadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:43.215708 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c4dd77-f9ec-467f-91cb-d4bdad51dadb None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:43.215990 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c4dd77-f9ec-467f-91cb-d4bdad51dadb None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:43.216263 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c4dd77-f9ec-467f-91cb-d4bdad51dadb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:43.281225 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c4dd77-f9ec-467f-91cb-d4bdad51dadb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:43.282748 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c4dd77-f9ec-467f-91cb-d4bdad51dadb None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:43.293424 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcaacbf-df3d-4f64-bcd3-d0436fb59851 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:43.325706 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e02241-a5a1-47a3-90e6-83e194b2c0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:43.326295 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e02241-a5a1-47a3-90e6-83e194b2c0b9 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:43.326577 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e02241-a5a1-47a3-90e6-83e194b2c0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:43.326849 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e02241-a5a1-47a3-90e6-83e194b2c0b9 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:43.327582 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e02241-a5a1-47a3-90e6-83e194b2c0b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:43:43.327890 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e02241-a5a1-47a3-90e6-83e194b2c0b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:43.329286 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e02241-a5a1-47a3-90e6-83e194b2c0b9 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:43.341961 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 7/15] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:43:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:43:43.347060 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e02241-a5a1-47a3-90e6-83e194b2c0b9 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:43.380657 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 9/16] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:43:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 06 14:43:43.386429 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c4dd77-f9ec-467f-91cb-d4bdad51dadb None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:43.444190 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e02241-a5a1-47a3-90e6-83e194b2c0b9 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:43.477473 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2120456-09af-4746-805f-8dfdd83b85cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:43.478931 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2120456-09af-4746-805f-8dfdd83b85cb None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:43.479242 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2120456-09af-4746-805f-8dfdd83b85cb None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:43.479521 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2120456-09af-4746-805f-8dfdd83b85cb None admin] PATH_INFO: `/v3/users/66f1687219494ee0a402084110e075fd` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:43.480797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2120456-09af-4746-805f-8dfdd83b85cb None admin] RBAC: Authorizing `identity:get_user(user_id=66f1687219494ee0a402084110e075fd)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:43.482295 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2120456-09af-4746-805f-8dfdd83b85cb None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:43.482840 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c4dd77-f9ec-467f-91cb-d4bdad51dadb None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:43.484025 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 8/17] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:43:43 2022] GET /identity/v3/users/66f1687219494ee0a402084110e075fd => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:43:43.489383 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2120456-09af-4746-805f-8dfdd83b85cb None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:43.516088 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1715cc-6b83-4e47-850e-e85c562f4026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:43.516923 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1715cc-6b83-4e47-850e-e85c562f4026 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:43.517208 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1715cc-6b83-4e47-850e-e85c562f4026 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:43.517480 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1715cc-6b83-4e47-850e-e85c562f4026 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:43.594417 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2120456-09af-4746-805f-8dfdd83b85cb None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:43.628683 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af152d6e-d6d9-41bc-ac4b-fd6f3bb8fa49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:43.629489 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af152d6e-d6d9-41bc-ac4b-fd6f3bb8fa49 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:43.629776 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af152d6e-d6d9-41bc-ac4b-fd6f3bb8fa49 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:43.630052 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af152d6e-d6d9-41bc-ac4b-fd6f3bb8fa49 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:43.631258 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af152d6e-d6d9-41bc-ac4b-fd6f3bb8fa49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:43.632754 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af152d6e-d6d9-41bc-ac4b-fd6f3bb8fa49 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:43.634874 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 9/18] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:43:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:43:43.641174 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af152d6e-d6d9-41bc-ac4b-fd6f3bb8fa49 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:43.644201 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1715cc-6b83-4e47-850e-e85c562f4026 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:43.645918 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1715cc-6b83-4e47-850e-e85c562f4026 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:43.671139 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-6f1715cc-6b83-4e47-850e-e85c562f4026 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:43:43.671139 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:43:43.671139 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:43:43.671139 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:43:43.671139 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:43:43.671139 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:43:43.671139 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:43:43.671139 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:43:43.671139 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:43:43.671139 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:43:43.671139 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:43:43.671139 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:43:43.671139 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:43:43.671139 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:43:43.671139 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:43:43.671997 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:43:43.671997 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:43:43.671997 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:43:43.671997 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:43:43.671997 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:43:43.671997 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:43:43.671997 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:43:43.671997 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:43:43.671997 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:43:43.671997 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:43:43.671997 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:43:43.671997 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:43:43.671997 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:43:43.671997 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:43:43.671997 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:43:43.671997 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:43:43.671997 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:43:43.672882 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:43:43.672882 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:43:43.672882 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:43:43.672882 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:43:43.672882 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 10/19] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:43:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:43:43.678273 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1715cc-6b83-4e47-850e-e85c562f4026 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:43.739340 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af152d6e-d6d9-41bc-ac4b-fd6f3bb8fa49 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:43.773445 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1715cc-6b83-4e47-850e-e85c562f4026 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:43.775398 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d69044d-e580-4448-bff6-454428e70241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:43.776549 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d69044d-e580-4448-bff6-454428e70241 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:43.777102 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d69044d-e580-4448-bff6-454428e70241 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:43.777605 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d69044d-e580-4448-bff6-454428e70241 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:43.779259 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d69044d-e580-4448-bff6-454428e70241 None admin] RBAC: Adding query filter params (role.id=6dcd35d1a83748759d6ef9f026077835, scope.domain.id=default, user.id=66f1687219494ee0a402084110e075fd) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:43:43.779838 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d69044d-e580-4448-bff6-454428e70241 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:43.782879 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d69044d-e580-4448-bff6-454428e70241 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:43.791475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 10/20] 10.222.0.23 () {56 vars in 1322 bytes} [Fri May 6 14:43:43 2022] GET /identity/v3/role_assignments?user.id=66f1687219494ee0a402084110e075fd&scope.domain.id=default&role.id=6dcd35d1a83748759d6ef9f026077835 => generated 233 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:43:43.809360 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf263db0-ae1f-46ba-8a35-1835378b2670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:43.810052 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf263db0-ae1f-46ba-8a35-1835378b2670 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:43.810339 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf263db0-ae1f-46ba-8a35-1835378b2670 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:43.810618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf263db0-ae1f-46ba-8a35-1835378b2670 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:43.811372 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf263db0-ae1f-46ba-8a35-1835378b2670 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:43:43.811681 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf263db0-ae1f-46ba-8a35-1835378b2670 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:43.813153 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf263db0-ae1f-46ba-8a35-1835378b2670 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:43.832864 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 11/21] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:43:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:43:46.782963 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01429fae-3e10-4847-8b51-e82baeaee409 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:46.783469 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01429fae-3e10-4847-8b51-e82baeaee409 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:46.783574 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01429fae-3e10-4847-8b51-e82baeaee409 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:46.784901 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 11/22] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:43:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:43:46.790878 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5247c-102d-4938-bc3e-128c71b0b166 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:46.791371 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5247c-102d-4938-bc3e-128c71b0b166 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:46.791447 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5247c-102d-4938-bc3e-128c71b0b166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:46.816113 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-fde5247c-102d-4938-bc3e-128c71b0b166 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:43:46.817363 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-fde5247c-102d-4938-bc3e-128c71b0b166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:43:46.912639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6261561b-96fe-4961-8d4b-d4c93be26aae None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:46.912639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6261561b-96fe-4961-8d4b-d4c93be26aae None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:46.912639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6261561b-96fe-4961-8d4b-d4c93be26aae None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:46.913190 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 12/23] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:43:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:43:46.918425 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667ab91-ba33-4566-b8cb-c21efdd48318 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:46.918964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667ab91-ba33-4566-b8cb-c21efdd48318 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:46.919459 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667ab91-ba33-4566-b8cb-c21efdd48318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:46.921998 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 12/24] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:43:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:43:46.948116 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde5247c-102d-4938-bc3e-128c71b0b166 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:46.953823 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-0667ab91-ba33-4566-b8cb-c21efdd48318 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:43:46.955051 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-0667ab91-ba33-4566-b8cb-c21efdd48318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:43:47.025489 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 13/25] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:43:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 06 14:43:47.036704 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f2dc2-193d-4afc-8f0c-b5db4c92af1e None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:47.037181 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f2dc2-193d-4afc-8f0c-b5db4c92af1e None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:47.037606 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f2dc2-193d-4afc-8f0c-b5db4c92af1e None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:47.039202 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 14/26] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:43:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:43:47.045224 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3cd42-c891-464d-a6db-dcdea7f30a9c None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:47.045802 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3cd42-c891-464d-a6db-dcdea7f30a9c None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:47.046311 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3cd42-c891-464d-a6db-dcdea7f30a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:47.057523 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde5247c-102d-4938-bc3e-128c71b0b166 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:47.072722 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-eef3cd42-c891-464d-a6db-dcdea7f30a9c None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:43:47.074616 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-eef3cd42-c891-464d-a6db-dcdea7f30a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:43:47.097901 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c6cf3-76dd-4f1b-9425-739e24362be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:47.098876 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c6cf3-76dd-4f1b-9425-739e24362be1 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:47.099257 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c6cf3-76dd-4f1b-9425-739e24362be1 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:47.099927 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c6cf3-76dd-4f1b-9425-739e24362be1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:47.113086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392c6cf3-76dd-4f1b-9425-739e24362be1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:43:47.125737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-392c6cf3-76dd-4f1b-9425-739e24362be1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:43:47.125737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:43:47.125737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:43:47.125737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:43:47.125737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:43:47.125737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:43:47.125737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:43:47.125737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:43:47.125737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:43:47.125737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:43:47.125737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:43:47.125737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:43:47.125737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:43:47.125737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:43:47.125737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:43:47.125737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:43:47.126641 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:43:47.126641 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:43:47.126641 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:43:47.126641 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:43:47.126641 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:43:47.126641 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:43:47.126641 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:43:47.126641 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:43:47.126641 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:43:47.126641 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:43:47.126641 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:43:47.126641 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:43:47.126641 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:43:47.126641 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:43:47.126641 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:43:47.126641 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:43:47.127393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:43:47.127393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:43:47.127393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:43:47.127393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:43:47.127393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:43:47.127393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:43:47.127393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:43:47.127393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:43:47.127393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:43:47.127393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:43:47.127393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:43:47.127393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:43:47.132867 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 13/27] 10.222.0.23 () {56 vars in 1096 bytes} [Fri May 6 14:43:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:43:47.134007 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c6cf3-76dd-4f1b-9425-739e24362be1 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:47.161910 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 15/28] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:43:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 06 14:43:47.163127 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef3cd42-c891-464d-a6db-dcdea7f30a9c None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:47.257994 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c6cf3-76dd-4f1b-9425-739e24362be1 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:47.269817 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef3cd42-c891-464d-a6db-dcdea7f30a9c None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:47.298915 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9360b5ed-1aca-4d66-a0fe-31b317dba343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:47.299644 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9360b5ed-1aca-4d66-a0fe-31b317dba343 None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:47.299932 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9360b5ed-1aca-4d66-a0fe-31b317dba343 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:47.300212 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9360b5ed-1aca-4d66-a0fe-31b317dba343 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:47.300987 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9360b5ed-1aca-4d66-a0fe-31b317dba343 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:47.302367 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9360b5ed-1aca-4d66-a0fe-31b317dba343 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:47.311511 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f81cbee-296f-4a0e-beb9-81f03997626b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:47.312662 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81cbee-296f-4a0e-beb9-81f03997626b None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:47.313190 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81cbee-296f-4a0e-beb9-81f03997626b None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:47.313744 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81cbee-296f-4a0e-beb9-81f03997626b None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:47.314732 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f81cbee-296f-4a0e-beb9-81f03997626b None admin] RBAC: Adding query filter params (name=admin) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:43:47.315248 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f81cbee-296f-4a0e-beb9-81f03997626b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:47.316882 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f81cbee-296f-4a0e-beb9-81f03997626b None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:47.327125 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 16/29] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:43:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:43:47.328227 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f81cbee-296f-4a0e-beb9-81f03997626b None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:47.434126 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 14/30] 10.222.0.23 () {60 vars in 1131 bytes} [Fri May 6 14:43:47 2022] POST /identity/v3/roles => generated 219 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:43:47.440630 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9360b5ed-1aca-4d66-a0fe-31b317dba343 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:47.446724 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f81cbee-296f-4a0e-beb9-81f03997626b None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:47.490201 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc71449-c83f-4beb-a735-07ccd2c5490c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:47.490888 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc71449-c83f-4beb-a735-07ccd2c5490c None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:47.491250 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc71449-c83f-4beb-a735-07ccd2c5490c None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:47.493963 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc71449-c83f-4beb-a735-07ccd2c5490c None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:47.493963 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc71449-c83f-4beb-a735-07ccd2c5490c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:47.495071 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc71449-c83f-4beb-a735-07ccd2c5490c None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:47.594682 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9360b5ed-1aca-4d66-a0fe-31b317dba343 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:47.649381 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f82359-95b7-4f39-870a-05fc87c2e633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:47.651167 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f82359-95b7-4f39-870a-05fc87c2e633 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:47.651470 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f82359-95b7-4f39-870a-05fc87c2e633 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:47.651754 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f82359-95b7-4f39-870a-05fc87c2e633 None admin] PATH_INFO: `/v3/users/66f1687219494ee0a402084110e075fd` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:47.657916 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f82359-95b7-4f39-870a-05fc87c2e633 None admin] RBAC: Authorizing `identity:get_user(user_id=66f1687219494ee0a402084110e075fd)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:47.659528 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f82359-95b7-4f39-870a-05fc87c2e633 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:47.661272 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 15/31] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:43:47 2022] GET /identity/v3/users/66f1687219494ee0a402084110e075fd => generated 247 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:43:47.662603 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5de7bf-7d37-4de0-a9fb-b74db9042c2b None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:47.662890 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5de7bf-7d37-4de0-a9fb-b74db9042c2b None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:47.663167 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5de7bf-7d37-4de0-a9fb-b74db9042c2b None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:47.664383 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 16/32] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:43:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:43:47.671366 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5de7bf-7d37-4de0-a9fb-b74db9042c2b None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:47.686581 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 17/33] 10.222.0.23 () {60 vars in 1131 bytes} [Fri May 6 14:43:47 2022] POST /identity/v3/roles => generated 225 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:43:47.688418 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb5cbd-4152-445b-8b84-2e0559682630 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:47.688790 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb5cbd-4152-445b-8b84-2e0559682630 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:47.689085 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb5cbd-4152-445b-8b84-2e0559682630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:47.708411 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-8efb5cbd-4152-445b-8b84-2e0559682630 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:43:47.710072 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-8efb5cbd-4152-445b-8b84-2e0559682630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:43:47.790930 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 18/34] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:43:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:43:47.816152 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efb5cbd-4152-445b-8b84-2e0559682630 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:47.857529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5de7bf-7d37-4de0-a9fb-b74db9042c2b None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:47.900565 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712b7920-0e17-4f17-844e-1923a311a60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:47.901405 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712b7920-0e17-4f17-844e-1923a311a60d None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:47.901696 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712b7920-0e17-4f17-844e-1923a311a60d None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:47.901978 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712b7920-0e17-4f17-844e-1923a311a60d None admin] PATH_INFO: `/v3/domains/default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:47.903285 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712b7920-0e17-4f17-844e-1923a311a60d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:47.904816 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712b7920-0e17-4f17-844e-1923a311a60d None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:47.906820 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 17/35] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:43:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:43:47.918006 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712b7920-0e17-4f17-844e-1923a311a60d None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:47.932091 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efb5cbd-4152-445b-8b84-2e0559682630 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:47.976397 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d2628-bf26-4ccb-b51f-af3c2364c311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:47.977317 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d2628-bf26-4ccb-b51f-af3c2364c311 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:47.977627 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d2628-bf26-4ccb-b51f-af3c2364c311 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:47.977948 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d2628-bf26-4ccb-b51f-af3c2364c311 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:47.992222 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d2628-bf26-4ccb-b51f-af3c2364c311 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:47.998020 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d2628-bf26-4ccb-b51f-af3c2364c311 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:48.022439 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-3e7d2628-bf26-4ccb-b51f-af3c2364c311 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:43:48.022439 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:43:48.022439 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:43:48.022439 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:43:48.022439 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:43:48.022439 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:43:48.022439 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:43:48.022439 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:43:48.022439 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:43:48.022439 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:43:48.022439 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:43:48.022439 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:43:48.022439 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:43:48.022439 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:43:48.022439 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:43:48.023321 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:43:48.023321 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:43:48.023321 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:43:48.023321 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:43:48.023321 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:43:48.023321 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:43:48.023321 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:43:48.023321 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:43:48.023321 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:43:48.023321 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:43:48.023321 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:43:48.023321 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:43:48.023321 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:43:48.023321 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:43:48.023321 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:43:48.023321 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:43:48.023321 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:43:48.024125 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:43:48.024125 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:43:48.024125 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:43:48.024125 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:43:48.024125 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 19/36] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:43:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:43:48.030392 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d2628-bf26-4ccb-b51f-af3c2364c311 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:48.048129 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712b7920-0e17-4f17-844e-1923a311a60d None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:48.085736 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf366b-9c07-48a7-a5dd-878cda0194af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:48.087567 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf366b-9c07-48a7-a5dd-878cda0194af None admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:48.087567 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf366b-9c07-48a7-a5dd-878cda0194af None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:48.087567 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf366b-9c07-48a7-a5dd-878cda0194af None admin] PATH_INFO: `/v3/domains/default/users/66f1687219494ee0a402084110e075fd/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:48.089920 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaf366b-9c07-48a7-a5dd-878cda0194af None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=66f1687219494ee0a402084110e075fd, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:48.091208 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaf366b-9c07-48a7-a5dd-878cda0194af None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:48.112248 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 18/37] 10.222.0.23 () {58 vars in 1361 bytes} [Fri May 6 14:43:47 2022] PUT /identity/v3/domains/default/users/66f1687219494ee0a402084110e075fd/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 14:43:48.113969 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773f6e2-682a-4418-baa3-3af6c1552dec None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:48.114556 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773f6e2-682a-4418-baa3-3af6c1552dec None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:48.115060 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773f6e2-682a-4418-baa3-3af6c1552dec None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:48.116854 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 19/38] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:43:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:43:48.124467 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96000d73-c9f4-4c3f-b543-9ec42fff7276 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:48.125073 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96000d73-c9f4-4c3f-b543-9ec42fff7276 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:48.125744 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96000d73-c9f4-4c3f-b543-9ec42fff7276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:48.150141 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-96000d73-c9f4-4c3f-b543-9ec42fff7276 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:43:48.153246 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-96000d73-c9f4-4c3f-b543-9ec42fff7276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:43:48.170214 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d2628-bf26-4ccb-b51f-af3c2364c311 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:48.210061 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd71e6-4871-42cd-befc-fee8970ac53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:48.210949 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd71e6-4871-42cd-befc-fee8970ac53d None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:48.211346 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd71e6-4871-42cd-befc-fee8970ac53d None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:48.211767 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd71e6-4871-42cd-befc-fee8970ac53d None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:48.212802 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bd71e6-4871-42cd-befc-fee8970ac53d None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:43:48.213237 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bd71e6-4871-42cd-befc-fee8970ac53d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:48.214862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bd71e6-4871-42cd-befc-fee8970ac53d None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:48.236046 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 20/39] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:43:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:43:48.242080 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd71e6-4871-42cd-befc-fee8970ac53d None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:48.253969 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 20/40] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:43:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:43:48.283361 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96000d73-c9f4-4c3f-b543-9ec42fff7276 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:48.387159 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd71e6-4871-42cd-befc-fee8970ac53d None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:48.408717 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96000d73-c9f4-4c3f-b543-9ec42fff7276 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:48.422095 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338f6d02-6bcb-46b3-a01c-d58bb31b10f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:48.422769 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f6d02-6bcb-46b3-a01c-d58bb31b10f9 None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:48.423057 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f6d02-6bcb-46b3-a01c-d58bb31b10f9 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:48.423336 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f6d02-6bcb-46b3-a01c-d58bb31b10f9 None admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:48.424072 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338f6d02-6bcb-46b3-a01c-d58bb31b10f9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:48.425550 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338f6d02-6bcb-46b3-a01c-d58bb31b10f9 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:48.444756 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6fd7ba-df6f-461b-b023-810dea09eb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:48.445675 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6fd7ba-df6f-461b-b023-810dea09eb96 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:48.446074 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6fd7ba-df6f-461b-b023-810dea09eb96 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:48.446506 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6fd7ba-df6f-461b-b023-810dea09eb96 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:48.447880 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6fd7ba-df6f-461b-b023-810dea09eb96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:48.449393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6fd7ba-df6f-461b-b023-810dea09eb96 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:48.455586 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 21/41] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:43:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:43:48.455586 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c51fe2d-bca1-4a8f-9f70-4a169be34db6 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:48.455586 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c51fe2d-bca1-4a8f-9f70-4a169be34db6 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:48.455586 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c51fe2d-bca1-4a8f-9f70-4a169be34db6 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:48.455586 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 22/42] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:43:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:43:48.458725 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 21/43] 10.222.0.23 () {60 vars in 1141 bytes} [Fri May 6 14:43:48 2022] POST /identity/v3/projects => generated 301 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 14:43:48.460568 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c51fe2d-bca1-4a8f-9f70-4a169be34db6 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:48.460719 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19d6b1-8a7c-4512-85c1-63a121145ad9 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:48.460719 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19d6b1-8a7c-4512-85c1-63a121145ad9 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:48.460845 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19d6b1-8a7c-4512-85c1-63a121145ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:48.480672 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-ed19d6b1-8a7c-4512-85c1-63a121145ad9 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:43:48.481916 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ed19d6b1-8a7c-4512-85c1-63a121145ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:43:48.582634 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 22/44] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:43:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:43:48.597014 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c51fe2d-bca1-4a8f-9f70-4a169be34db6 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:48.605953 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed19d6b1-8a7c-4512-85c1-63a121145ad9 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:48.649589 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27276f56-0bb2-4bc6-8705-829f3c26435e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:48.650521 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27276f56-0bb2-4bc6-8705-829f3c26435e None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:48.651571 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27276f56-0bb2-4bc6-8705-829f3c26435e None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:48.652005 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27276f56-0bb2-4bc6-8705-829f3c26435e None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:48.652910 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27276f56-0bb2-4bc6-8705-829f3c26435e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:48.655440 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27276f56-0bb2-4bc6-8705-829f3c26435e None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:48.690513 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 23/45] 10.222.0.23 () {60 vars in 1141 bytes} [Fri May 6 14:43:48 2022] POST /identity/v3/projects => generated 312 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:43:48.730682 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed19d6b1-8a7c-4512-85c1-63a121145ad9 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:48.770681 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750e0059-a674-447f-a668-8e6c184fd912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:48.771474 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e0059-a674-447f-a668-8e6c184fd912 None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:48.771761 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e0059-a674-447f-a668-8e6c184fd912 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:48.772040 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e0059-a674-447f-a668-8e6c184fd912 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:48.772985 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750e0059-a674-447f-a668-8e6c184fd912 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:48.774452 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750e0059-a674-447f-a668-8e6c184fd912 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:48.789708 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 23/46] 10.222.0.23 () {60 vars in 1131 bytes} [Fri May 6 14:43:48 2022] POST /identity/v3/roles => generated 223 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 14:43:50.439184 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909bbd7-37af-441d-bbaf-5c207e19a019 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:50.439692 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909bbd7-37af-441d-bbaf-5c207e19a019 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:50.439692 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909bbd7-37af-441d-bbaf-5c207e19a019 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:50.441151 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 24/47] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:43:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:43:50.446400 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f44239-8d84-4626-adb8-75dd3106aba5 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:50.446904 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f44239-8d84-4626-adb8-75dd3106aba5 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:50.447091 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f44239-8d84-4626-adb8-75dd3106aba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:50.464595 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-f5f44239-8d84-4626-adb8-75dd3106aba5 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:43:50.465803 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-f5f44239-8d84-4626-adb8-75dd3106aba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:43:50.532161 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 24/48] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:43:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:43:50.554712 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c909bbd7-37af-441d-bbaf-5c207e19a019 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:50.652738 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c909bbd7-37af-441d-bbaf-5c207e19a019 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:50.688924 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d1539-db8a-4a33-856d-43acff5f225d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:50.689769 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d1539-db8a-4a33-856d-43acff5f225d None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:50.690002 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d1539-db8a-4a33-856d-43acff5f225d None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:50.690275 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d1539-db8a-4a33-856d-43acff5f225d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:50.700644 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777d1539-db8a-4a33-856d-43acff5f225d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:43:50.708129 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-777d1539-db8a-4a33-856d-43acff5f225d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:43:50.708129 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:43:50.708129 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:43:50.708129 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:43:50.708129 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:43:50.708129 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:43:50.708129 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:43:50.708129 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:43:50.708129 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:43:50.708129 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:43:50.708129 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:43:50.708129 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:43:50.708129 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:43:50.708129 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:43:50.708129 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:43:50.708129 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:43:50.709209 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:43:50.709209 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:43:50.709209 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:43:50.709209 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:43:50.709209 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:43:50.709209 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:43:50.709209 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:43:50.709209 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:43:50.709209 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:43:50.709209 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:43:50.709209 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:43:50.709209 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:43:50.709209 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:43:50.709209 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:43:50.709209 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:43:50.709209 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:43:50.710142 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:43:50.710142 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:43:50.710142 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:43:50.710142 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:43:50.710142 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:43:50.710142 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:43:50.710142 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:43:50.710142 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:43:50.710142 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:43:50.710142 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:43:50.710142 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:43:50.710142 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:43:50.710142 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 25/49] 10.222.0.23 () {56 vars in 1096 bytes} [Fri May 6 14:43:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:43:50.717137 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f44239-8d84-4626-adb8-75dd3106aba5 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:50.817650 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f44239-8d84-4626-adb8-75dd3106aba5 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:50.850826 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8177afe9-49d8-4d5e-9f07-ac386cee1c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:50.851499 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8177afe9-49d8-4d5e-9f07-ac386cee1c24 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:50.851785 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8177afe9-49d8-4d5e-9f07-ac386cee1c24 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:50.852068 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8177afe9-49d8-4d5e-9f07-ac386cee1c24 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:50.852904 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8177afe9-49d8-4d5e-9f07-ac386cee1c24 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:43:50.853292 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8177afe9-49d8-4d5e-9f07-ac386cee1c24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:50.854592 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8177afe9-49d8-4d5e-9f07-ac386cee1c24 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:50.862792 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 25/50] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:43:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:43:50.868955 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d1539-db8a-4a33-856d-43acff5f225d None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:50.957972 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c0314-de18-4efb-88d1-64f446774c23 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:50.959312 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c0314-de18-4efb-88d1-64f446774c23 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:50.959312 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c0314-de18-4efb-88d1-64f446774c23 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:50.959821 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 26/51] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:43:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:43:50.964874 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828c8ae2-82cd-4856-ba12-4fad354363d8 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:50.965167 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828c8ae2-82cd-4856-ba12-4fad354363d8 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:50.965450 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828c8ae2-82cd-4856-ba12-4fad354363d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:50.965901 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d1539-db8a-4a33-856d-43acff5f225d None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:50.982516 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-828c8ae2-82cd-4856-ba12-4fad354363d8 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:43:50.983838 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-828c8ae2-82cd-4856-ba12-4fad354363d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:43:50.999908 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b457ae-42fa-480b-b6fe-c2cfa488d3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:51.000821 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b457ae-42fa-480b-b6fe-c2cfa488d3d5 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:51.001109 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b457ae-42fa-480b-b6fe-c2cfa488d3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:51.001388 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b457ae-42fa-480b-b6fe-c2cfa488d3d5 None admin] PATH_INFO: `/v3/users/66f1687219494ee0a402084110e075fd` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:51.002581 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b457ae-42fa-480b-b6fe-c2cfa488d3d5 None admin] RBAC: Authorizing `identity:get_user(user_id=66f1687219494ee0a402084110e075fd)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:51.004023 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b457ae-42fa-480b-b6fe-c2cfa488d3d5 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:51.005684 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 26/52] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:43:50 2022] GET /identity/v3/users/66f1687219494ee0a402084110e075fd => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:43:51.011010 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b457ae-42fa-480b-b6fe-c2cfa488d3d5 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:51.049825 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 27/53] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:43:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 06 14:43:51.073205 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828c8ae2-82cd-4856-ba12-4fad354363d8 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:51.106143 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b457ae-42fa-480b-b6fe-c2cfa488d3d5 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:51.138972 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec80599-dcc2-4ab9-9b79-91c8aa57524d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:51.139761 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec80599-dcc2-4ab9-9b79-91c8aa57524d None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:51.140049 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec80599-dcc2-4ab9-9b79-91c8aa57524d None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:51.140331 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec80599-dcc2-4ab9-9b79-91c8aa57524d None admin] PATH_INFO: `/v3/domains/default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:51.141536 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec80599-dcc2-4ab9-9b79-91c8aa57524d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:51.142996 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec80599-dcc2-4ab9-9b79-91c8aa57524d None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:51.144625 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 27/54] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:43:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:43:51.150385 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec80599-dcc2-4ab9-9b79-91c8aa57524d None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:51.167727 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828c8ae2-82cd-4856-ba12-4fad354363d8 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:51.200751 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac9ff2b-6c31-4173-b0cb-03728707fd36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:51.201546 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac9ff2b-6c31-4173-b0cb-03728707fd36 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:51.201835 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac9ff2b-6c31-4173-b0cb-03728707fd36 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:51.202114 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac9ff2b-6c31-4173-b0cb-03728707fd36 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:51.203291 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac9ff2b-6c31-4173-b0cb-03728707fd36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:51.204794 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac9ff2b-6c31-4173-b0cb-03728707fd36 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:51.206395 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 28/55] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:43:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:43:51.212221 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac9ff2b-6c31-4173-b0cb-03728707fd36 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:51.247680 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec80599-dcc2-4ab9-9b79-91c8aa57524d None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:51.279293 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319e6438-216f-41e9-83e9-528aa233ef5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:51.279947 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e6438-216f-41e9-83e9-528aa233ef5a None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:51.280235 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e6438-216f-41e9-83e9-528aa233ef5a None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:51.280516 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e6438-216f-41e9-83e9-528aa233ef5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:51.281636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319e6438-216f-41e9-83e9-528aa233ef5a None admin] RBAC: Adding query filter params (role.id=6dcd35d1a83748759d6ef9f026077835, scope.domain.id=default, user.id=66f1687219494ee0a402084110e075fd) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:43:51.281951 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319e6438-216f-41e9-83e9-528aa233ef5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:51.283374 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319e6438-216f-41e9-83e9-528aa233ef5a None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:51.291705 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 28/56] 10.222.0.23 () {56 vars in 1322 bytes} [Fri May 6 14:43:51 2022] GET /identity/v3/role_assignments?user.id=66f1687219494ee0a402084110e075fd&scope.domain.id=default&role.id=6dcd35d1a83748759d6ef9f026077835 => generated 530 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:43:51.307811 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac9ff2b-6c31-4173-b0cb-03728707fd36 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:51.344254 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7913aea-d4db-4294-9a25-fed73f199ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:51.344962 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7913aea-d4db-4294-9a25-fed73f199ae3 None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:51.345250 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7913aea-d4db-4294-9a25-fed73f199ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:51.345531 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7913aea-d4db-4294-9a25-fed73f199ae3 None admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:51.346256 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7913aea-d4db-4294-9a25-fed73f199ae3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:51.347751 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7913aea-d4db-4294-9a25-fed73f199ae3 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:51.376993 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 29/57] 10.222.0.23 () {60 vars in 1140 bytes} [Fri May 6 14:43:51 2022] POST /identity/v3/projects => generated 298 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 14:43:53.630007 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a904b-23ea-4956-aa19-c58ed8e41006 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:53.630541 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a904b-23ea-4956-aa19-c58ed8e41006 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:53.630653 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a904b-23ea-4956-aa19-c58ed8e41006 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:53.631942 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 29/58] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:43:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:43:53.637502 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a71cc-de38-4e47-83ec-c699f66719ae None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:53.637883 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a71cc-de38-4e47-83ec-c699f66719ae None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:53.638074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a71cc-de38-4e47-83ec-c699f66719ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:53.655018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-d81a71cc-de38-4e47-83ec-c699f66719ae None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:43:53.656454 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d81a71cc-de38-4e47-83ec-c699f66719ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:43:53.719337 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 30/59] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:43:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 06 14:43:53.744007 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351a904b-23ea-4956-aa19-c58ed8e41006 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:53.841050 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351a904b-23ea-4956-aa19-c58ed8e41006 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:53.872946 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292cee54-aa08-4b4d-9786-40ecad26e7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:53.873730 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292cee54-aa08-4b4d-9786-40ecad26e7fb None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:53.874017 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292cee54-aa08-4b4d-9786-40ecad26e7fb None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:53.874294 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292cee54-aa08-4b4d-9786-40ecad26e7fb None admin] PATH_INFO: `/v3/domains/default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:53.875462 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292cee54-aa08-4b4d-9786-40ecad26e7fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:53.876932 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292cee54-aa08-4b4d-9786-40ecad26e7fb None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:53.878532 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 30/60] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:43:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:43:53.884244 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a71cc-de38-4e47-83ec-c699f66719ae None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:53.986805 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a71cc-de38-4e47-83ec-c699f66719ae None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:54.018576 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b3d595-8e72-4118-b5fb-d760ff9afabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:54.019249 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3d595-8e72-4118-b5fb-d760ff9afabf None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:54.019536 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3d595-8e72-4118-b5fb-d760ff9afabf None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:54.019819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3d595-8e72-4118-b5fb-d760ff9afabf None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:54.020528 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b3d595-8e72-4118-b5fb-d760ff9afabf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:54.022007 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b3d595-8e72-4118-b5fb-d760ff9afabf None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:54.046845 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 31/61] 10.222.0.23 () {60 vars in 1132 bytes} [Fri May 6 14:43:53 2022] POST /identity/v3/users => generated 275 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:43:57.235800 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82872ea-0072-460f-ad63-72859991679e None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:57.236824 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82872ea-0072-460f-ad63-72859991679e None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:57.237305 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82872ea-0072-460f-ad63-72859991679e None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:57.238866 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 31/62] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:43:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:43:57.245143 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfca0ad-b9eb-4660-96c1-72ba4caf1a68 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:57.245994 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfca0ad-b9eb-4660-96c1-72ba4caf1a68 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:57.246476 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfca0ad-b9eb-4660-96c1-72ba4caf1a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:57.269623 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-ebfca0ad-b9eb-4660-96c1-72ba4caf1a68 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:43:57.271165 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ebfca0ad-b9eb-4660-96c1-72ba4caf1a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:43:57.358474 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 32/63] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:43:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:43:57.392793 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82872ea-0072-460f-ad63-72859991679e None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:57.529017 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82872ea-0072-460f-ad63-72859991679e None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:57.583586 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c571bfe6-2865-48c7-ab2a-8d187e94bc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:57.584794 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571bfe6-2865-48c7-ab2a-8d187e94bc60 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:57.585281 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571bfe6-2865-48c7-ab2a-8d187e94bc60 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:57.585755 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571bfe6-2865-48c7-ab2a-8d187e94bc60 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:57.605638 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c571bfe6-2865-48c7-ab2a-8d187e94bc60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:43:57.613804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-c571bfe6-2865-48c7-ab2a-8d187e94bc60 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 06 14:43:57.613804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:43:57.613804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:43:57.613804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:43:57.613804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:43:57.613804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:43:57.613804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:43:57.613804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:43:57.613804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:43:57.613804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:43:57.613804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:43:57.613804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:43:57.613804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:43:57.613804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:43:57.613804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:43:57.613804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:43:57.614647 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:43:57.614647 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:43:57.614647 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:43:57.614647 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:43:57.614647 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:43:57.614647 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:43:57.614647 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:43:57.614647 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:43:57.614647 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:43:57.614647 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:43:57.614647 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:43:57.614647 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:43:57.614647 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:43:57.614647 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:43:57.614647 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:43:57.614647 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:43:57.615398 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:43:57.615398 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:43:57.615398 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:43:57.615398 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:43:57.615398 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:43:57.615398 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:43:57.615398 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:43:57.615398 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:43:57.615398 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:43:57.615398 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:43:57.615398 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 06 14:43:57.615398 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:43:57.617922 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 32/64] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:43:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:43:57.623748 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfca0ad-b9eb-4660-96c1-72ba4caf1a68 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:57.651277 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f2b48-6ced-4f45-9912-ddd78d95d988 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:57.651786 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f2b48-6ced-4f45-9912-ddd78d95d988 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:57.652242 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f2b48-6ced-4f45-9912-ddd78d95d988 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:57.653823 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 33/65] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:43:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:43:57.659538 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435874e9-d01d-424f-bc32-564966161daf None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:57.660019 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435874e9-d01d-424f-bc32-564966161daf None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:57.660446 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435874e9-d01d-424f-bc32-564966161daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:57.701673 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-435874e9-d01d-424f-bc32-564966161daf None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:43:57.703167 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-435874e9-d01d-424f-bc32-564966161daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:43:57.733389 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfca0ad-b9eb-4660-96c1-72ba4caf1a68 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:57.773277 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb7efb-8441-4892-9777-aac72a907824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:57.774405 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb7efb-8441-4892-9777-aac72a907824 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:57.775606 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb7efb-8441-4892-9777-aac72a907824 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:57.776300 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb7efb-8441-4892-9777-aac72a907824 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:57.778032 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebb7efb-8441-4892-9777-aac72a907824 None admin] RBAC: Adding query filter params (name=member) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:43:57.779188 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebb7efb-8441-4892-9777-aac72a907824 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:57.781659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebb7efb-8441-4892-9777-aac72a907824 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:57.803278 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 33/66] 10.222.0.23 () {56 vars in 1101 bytes} [Fri May 6 14:43:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:43:57.811191 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb7efb-8441-4892-9777-aac72a907824 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:57.853248 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 34/67] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:43:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 06 14:43:57.889413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435874e9-d01d-424f-bc32-564966161daf None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:57.983807 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb7efb-8441-4892-9777-aac72a907824 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:58.020136 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b7670-bebb-48ad-b8e8-5c8a9a934203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:58.021039 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b7670-bebb-48ad-b8e8-5c8a9a934203 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:58.021331 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b7670-bebb-48ad-b8e8-5c8a9a934203 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:58.021610 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b7670-bebb-48ad-b8e8-5c8a9a934203 None admin] PATH_INFO: `/v3/users/f26c0bc8ce6d4932bba691c04bec3ded` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:58.034460 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435874e9-d01d-424f-bc32-564966161daf None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:58.036518 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2b7670-bebb-48ad-b8e8-5c8a9a934203 None admin] RBAC: Authorizing `identity:get_user(user_id=f26c0bc8ce6d4932bba691c04bec3ded)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:58.038045 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2b7670-bebb-48ad-b8e8-5c8a9a934203 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:58.040136 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 34/68] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:43:57 2022] GET /identity/v3/users/f26c0bc8ce6d4932bba691c04bec3ded => generated 275 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:43:58.045710 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fece670-99dc-4989-beb5-b2c1ac11a2d6 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:58.045710 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fece670-99dc-4989-beb5-b2c1ac11a2d6 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:58.045710 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fece670-99dc-4989-beb5-b2c1ac11a2d6 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:58.045710 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 35/69] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:43:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:43:58.049541 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fece670-99dc-4989-beb5-b2c1ac11a2d6 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:58.081744 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6468326b-9337-40ec-b3de-c9bb4b0b0633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:58.082563 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468326b-9337-40ec-b3de-c9bb4b0b0633 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:58.082855 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468326b-9337-40ec-b3de-c9bb4b0b0633 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:58.083142 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468326b-9337-40ec-b3de-c9bb4b0b0633 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:58.094102 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6468326b-9337-40ec-b3de-c9bb4b0b0633 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:43:58.101876 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-6468326b-9337-40ec-b3de-c9bb4b0b0633 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 06 14:43:58.101876 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:43:58.101876 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:43:58.101876 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:43:58.101876 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:43:58.101876 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:43:58.101876 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:43:58.101876 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:43:58.101876 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:43:58.101876 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:43:58.101876 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:43:58.101876 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:43:58.101876 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:43:58.101876 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:43:58.101876 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:43:58.101876 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:43:58.103086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:43:58.103086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:43:58.103086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:43:58.103086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:43:58.103086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:43:58.103086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:43:58.103086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:43:58.103086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:43:58.103086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:43:58.103086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:43:58.103086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:43:58.103086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:43:58.103086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:43:58.103086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:43:58.103086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:43:58.103086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:43:58.103840 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:43:58.103840 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:43:58.103840 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:43:58.103840 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:43:58.103840 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:43:58.103840 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:43:58.103840 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:43:58.103840 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:43:58.103840 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:43:58.103840 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:43:58.103840 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 06 14:43:58.103840 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:43:58.103840 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 35/70] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:43:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:43:58.107613 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38aa406-bc5f-4b25-8564-adaa4301858a None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:58.108028 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38aa406-bc5f-4b25-8564-adaa4301858a None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:58.108318 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38aa406-bc5f-4b25-8564-adaa4301858a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:58.128584 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-d38aa406-bc5f-4b25-8564-adaa4301858a None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:43:58.129826 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-d38aa406-bc5f-4b25-8564-adaa4301858a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:43:58.167402 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fece670-99dc-4989-beb5-b2c1ac11a2d6 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:58.198094 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 36/71] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:43:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:43:58.199490 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc9e33-b9c1-49bd-a93a-d478ac465683 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:58.199830 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc9e33-b9c1-49bd-a93a-d478ac465683 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:58.200051 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc9e33-b9c1-49bd-a93a-d478ac465683 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:58.204686 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cbd18e-5f2e-4aaf-ac96-4cb6dc75cba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:58.204836 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 37/72] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:43:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:43:58.204909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cbd18e-5f2e-4aaf-ac96-4cb6dc75cba4 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:58.204909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cbd18e-5f2e-4aaf-ac96-4cb6dc75cba4 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:58.205049 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc9e33-b9c1-49bd-a93a-d478ac465683 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:58.205117 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cbd18e-5f2e-4aaf-ac96-4cb6dc75cba4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:58.283038 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cbd18e-5f2e-4aaf-ac96-4cb6dc75cba4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:58.284577 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cbd18e-5f2e-4aaf-ac96-4cb6dc75cba4 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:58.314599 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc9e33-b9c1-49bd-a93a-d478ac465683 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:58.347214 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84feca13-08db-4417-bbf1-5cd88be2ec58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:58.347886 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84feca13-08db-4417-bbf1-5cd88be2ec58 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:58.348176 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84feca13-08db-4417-bbf1-5cd88be2ec58 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:58.348458 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84feca13-08db-4417-bbf1-5cd88be2ec58 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:58.349250 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84feca13-08db-4417-bbf1-5cd88be2ec58 None admin] RBAC: Adding query filter params (name=member) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:43:58.349563 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84feca13-08db-4417-bbf1-5cd88be2ec58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:58.350959 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84feca13-08db-4417-bbf1-5cd88be2ec58 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:58.362439 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 38/73] 10.222.0.23 () {56 vars in 1101 bytes} [Fri May 6 14:43:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:43:58.363850 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9fcdc-9662-4963-98d2-2ba4414222e0 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:58.364151 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9fcdc-9662-4963-98d2-2ba4414222e0 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:58.364438 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9fcdc-9662-4963-98d2-2ba4414222e0 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:58.365843 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 39/74] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:43:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:43:58.369791 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d8acfc-d759-4001-bd9a-00dac28e8282 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:58.370081 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d8acfc-d759-4001-bd9a-00dac28e8282 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:58.370364 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d8acfc-d759-4001-bd9a-00dac28e8282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:58.381156 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 36/75] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:43:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 06 14:43:58.382698 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cbd18e-5f2e-4aaf-ac96-4cb6dc75cba4 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:58.387307 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-19d8acfc-d759-4001-bd9a-00dac28e8282 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:43:58.388496 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-19d8acfc-d759-4001-bd9a-00dac28e8282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:43:58.451992 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 40/76] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:43:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:43:58.464878 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d8acfc-d759-4001-bd9a-00dac28e8282 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:58.476818 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cbd18e-5f2e-4aaf-ac96-4cb6dc75cba4 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:58.510952 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2985d9-ab13-427f-bc6a-a89224bcb425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:58.511749 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2985d9-ab13-427f-bc6a-a89224bcb425 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:58.512038 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2985d9-ab13-427f-bc6a-a89224bcb425 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:58.512312 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2985d9-ab13-427f-bc6a-a89224bcb425 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:58.513514 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2985d9-ab13-427f-bc6a-a89224bcb425 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:58.514967 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2985d9-ab13-427f-bc6a-a89224bcb425 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:58.516572 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 37/77] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:43:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:43:58.517908 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ef8d5-287b-44b3-8256-e514059e57de None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:58.518203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ef8d5-287b-44b3-8256-e514059e57de None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:58.518486 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ef8d5-287b-44b3-8256-e514059e57de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:58.534354 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-b35ef8d5-287b-44b3-8256-e514059e57de None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:43:58.535615 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b35ef8d5-287b-44b3-8256-e514059e57de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:43:58.564220 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d8acfc-d759-4001-bd9a-00dac28e8282 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:58.597191 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14edb740-c770-4ebe-a303-77b3f5ce713c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:58.598064 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14edb740-c770-4ebe-a303-77b3f5ce713c None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:58.598343 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14edb740-c770-4ebe-a303-77b3f5ce713c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:58.598617 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14edb740-c770-4ebe-a303-77b3f5ce713c None admin] PATH_INFO: `/v3/users/f26c0bc8ce6d4932bba691c04bec3ded` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:58.599789 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14edb740-c770-4ebe-a303-77b3f5ce713c None admin] RBAC: Authorizing `identity:get_user(user_id=f26c0bc8ce6d4932bba691c04bec3ded)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:58.600626 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 38/78] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:43:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:43:58.601270 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14edb740-c770-4ebe-a303-77b3f5ce713c None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:58.601468 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35ef8d5-287b-44b3-8256-e514059e57de None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:58.607864 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 41/79] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:43:58 2022] GET /identity/v3/users/f26c0bc8ce6d4932bba691c04bec3ded => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:43:58.607864 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14edb740-c770-4ebe-a303-77b3f5ce713c None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:58.699331 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14edb740-c770-4ebe-a303-77b3f5ce713c None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:58.705218 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35ef8d5-287b-44b3-8256-e514059e57de None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:58.732219 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4cd6dd-21c5-44be-99a3-e41743e31ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:58.733051 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4cd6dd-21c5-44be-99a3-e41743e31ce7 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:58.733336 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4cd6dd-21c5-44be-99a3-e41743e31ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:58.733610 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4cd6dd-21c5-44be-99a3-e41743e31ce7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:58.737690 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e910c9c-8a81-4ff0-828d-ad087ed14ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:58.738550 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e910c9c-8a81-4ff0-828d-ad087ed14ee4 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:58.738834 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e910c9c-8a81-4ff0-828d-ad087ed14ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:58.739112 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e910c9c-8a81-4ff0-828d-ad087ed14ee4 None admin] PATH_INFO: `/v3/projects/92337f7cabcc48ff8ca007298d1c3d78` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:58.740292 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e910c9c-8a81-4ff0-828d-ad087ed14ee4 None admin] RBAC: Authorizing `identity:get_project(project_id=92337f7cabcc48ff8ca007298d1c3d78)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:58.741796 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e910c9c-8a81-4ff0-828d-ad087ed14ee4 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:58.743913 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4cd6dd-21c5-44be-99a3-e41743e31ce7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:43:58.744092 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 39/80] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:43:58 2022] GET /identity/v3/projects/92337f7cabcc48ff8ca007298d1c3d78 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:43:58.744871 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e910c9c-8a81-4ff0-828d-ad087ed14ee4 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:58.752337 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-6b4cd6dd-21c5-44be-99a3-e41743e31ce7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 06 14:43:58.752337 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:43:58.752337 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:43:58.752337 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:43:58.752337 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:43:58.752337 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:43:58.752337 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:43:58.752337 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:43:58.752337 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:43:58.752337 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:43:58.752337 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:43:58.752337 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:43:58.752337 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:43:58.752337 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:43:58.752337 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:43:58.752337 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:43:58.753259 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:43:58.753259 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:43:58.753259 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:43:58.753259 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:43:58.753259 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:43:58.753259 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:43:58.753259 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:43:58.753259 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:43:58.753259 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:43:58.753259 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:43:58.753259 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:43:58.753259 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:43:58.753259 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:43:58.753259 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:43:58.753259 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:43:58.753259 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:43:58.754083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:43:58.754083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:43:58.754083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:43:58.754083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:43:58.754083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:43:58.754083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:43:58.754083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:43:58.754083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:43:58.754083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:43:58.754083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:43:58.754083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 06 14:43:58.754083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:43:58.754083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 42/81] 10.222.0.23 () {56 vars in 1114 bytes} [Fri May 6 14:43:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:43:58.754705 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4cd6dd-21c5-44be-99a3-e41743e31ce7 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:58.839933 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e910c9c-8a81-4ff0-828d-ad087ed14ee4 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:58.854533 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4cd6dd-21c5-44be-99a3-e41743e31ce7 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:58.872591 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed23c5f-cf51-444a-aff3-be41dd238652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:58.873280 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed23c5f-cf51-444a-aff3-be41dd238652 None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:58.873560 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed23c5f-cf51-444a-aff3-be41dd238652 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:58.873831 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed23c5f-cf51-444a-aff3-be41dd238652 None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:58.874529 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed23c5f-cf51-444a-aff3-be41dd238652 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:58.875977 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed23c5f-cf51-444a-aff3-be41dd238652 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:58.890426 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867fc062-4896-4fd8-bfcf-c37a09aabb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:58.891032 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867fc062-4896-4fd8-bfcf-c37a09aabb27 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:58.891321 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867fc062-4896-4fd8-bfcf-c37a09aabb27 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:58.891600 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867fc062-4896-4fd8-bfcf-c37a09aabb27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:58.895078 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 40/82] 10.222.0.23 () {60 vars in 1132 bytes} [Fri May 6 14:43:58 2022] POST /identity/v3/users => generated 289 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:43:58.895896 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed23c5f-cf51-444a-aff3-be41dd238652 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:58.967615 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867fc062-4896-4fd8-bfcf-c37a09aabb27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:58.969161 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867fc062-4896-4fd8-bfcf-c37a09aabb27 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:59.005603 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed23c5f-cf51-444a-aff3-be41dd238652 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:59.042901 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc65afc-d65b-4890-851f-6fd1bdd4b0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:59.044346 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc65afc-d65b-4890-851f-6fd1bdd4b0e3 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:59.044346 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc65afc-d65b-4890-851f-6fd1bdd4b0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:59.044346 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc65afc-d65b-4890-851f-6fd1bdd4b0e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:59.055717 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc65afc-d65b-4890-851f-6fd1bdd4b0e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:43:59.063747 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-3cc65afc-d65b-4890-851f-6fd1bdd4b0e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:43:59.063747 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:43:59.063747 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:43:59.063747 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:43:59.063747 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:43:59.063747 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:43:59.063747 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:43:59.063747 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:43:59.063747 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:43:59.063747 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:43:59.063747 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:43:59.063747 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:43:59.063747 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:43:59.063747 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:43:59.063747 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:43:59.063747 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:43:59.064712 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:43:59.064712 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:43:59.064712 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:43:59.064712 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:43:59.064712 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:43:59.064712 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:43:59.064712 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:43:59.064712 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:43:59.064712 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:43:59.064712 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:43:59.064712 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:43:59.064712 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:43:59.064712 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:43:59.064712 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:43:59.064712 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:43:59.064712 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:43:59.065504 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:43:59.065504 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:43:59.065504 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:43:59.065504 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:43:59.065504 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:43:59.065504 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:43:59.065504 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:43:59.065504 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:43:59.065504 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:43:59.065504 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:43:59.065504 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:43:59.065504 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:43:59.065504 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 41/83] 10.222.0.23 () {56 vars in 1096 bytes} [Fri May 6 14:43:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:43:59.069781 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc65afc-d65b-4890-851f-6fd1bdd4b0e3 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:59.078410 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 43/84] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:43:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 06 14:43:59.079378 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867fc062-4896-4fd8-bfcf-c37a09aabb27 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:59.182787 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867fc062-4896-4fd8-bfcf-c37a09aabb27 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:59.187463 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc65afc-d65b-4890-851f-6fd1bdd4b0e3 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:59.230441 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c494328-fc2d-4c2a-a779-303a8b8b63f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:59.231177 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c494328-fc2d-4c2a-a779-303a8b8b63f5 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:59.231468 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c494328-fc2d-4c2a-a779-303a8b8b63f5 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:59.231576 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474d537b-2067-49b1-8e5b-9b096e243db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:59.231793 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c494328-fc2d-4c2a-a779-303a8b8b63f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:59.232531 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d537b-2067-49b1-8e5b-9b096e243db9 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:59.232764 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c494328-fc2d-4c2a-a779-303a8b8b63f5 None admin] RBAC: Adding query filter params (role.id=5f76e2e820b64d14b893c679427f4bed, scope.project.id=92337f7cabcc48ff8ca007298d1c3d78, user.id=f26c0bc8ce6d4932bba691c04bec3ded) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:43:59.232993 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c494328-fc2d-4c2a-a779-303a8b8b63f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:59.233348 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d537b-2067-49b1-8e5b-9b096e243db9 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:59.233763 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d537b-2067-49b1-8e5b-9b096e243db9 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:59.234479 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c494328-fc2d-4c2a-a779-303a8b8b63f5 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:59.234810 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474d537b-2067-49b1-8e5b-9b096e243db9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:43:59.235252 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474d537b-2067-49b1-8e5b-9b096e243db9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:59.236792 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474d537b-2067-49b1-8e5b-9b096e243db9 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:59.241503 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 42/85] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:43:59 2022] GET /identity/v3/role_assignments?user.id=f26c0bc8ce6d4932bba691c04bec3ded&scope.project.id=92337f7cabcc48ff8ca007298d1c3d78&role.id=5f76e2e820b64d14b893c679427f4bed => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:43:59.242855 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c494328-fc2d-4c2a-a779-303a8b8b63f5 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:59.246539 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 44/86] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:43:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:43:59.247632 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474d537b-2067-49b1-8e5b-9b096e243db9 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:59.359932 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474d537b-2067-49b1-8e5b-9b096e243db9 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:59.360559 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c494328-fc2d-4c2a-a779-303a8b8b63f5 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:59.397701 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93bc85-6af4-47f0-a1d4-69ffc0182171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:59.397945 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54482721-1197-445a-a4a1-19287b0f7f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:59.398554 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54482721-1197-445a-a4a1-19287b0f7f0c None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:59.398847 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54482721-1197-445a-a4a1-19287b0f7f0c None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:59.399200 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54482721-1197-445a-a4a1-19287b0f7f0c None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:59.399502 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93bc85-6af4-47f0-a1d4-69ffc0182171 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:59.399944 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93bc85-6af4-47f0-a1d4-69ffc0182171 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:59.400050 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54482721-1197-445a-a4a1-19287b0f7f0c None admin] RBAC: Adding query filter params (name=admin) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:43:59.400261 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54482721-1197-445a-a4a1-19287b0f7f0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:59.400582 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93bc85-6af4-47f0-a1d4-69ffc0182171 None admin] PATH_INFO: `/v3/projects/fd4139f9cd2643d3b04dfeda92c67546` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:59.401799 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54482721-1197-445a-a4a1-19287b0f7f0c None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:59.402170 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc93bc85-6af4-47f0-a1d4-69ffc0182171 None admin] RBAC: Authorizing `identity:get_project(project_id=fd4139f9cd2643d3b04dfeda92c67546)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:59.403781 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc93bc85-6af4-47f0-a1d4-69ffc0182171 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:59.405709 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 45/87] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:43:59 2022] GET /identity/v3/projects/fd4139f9cd2643d3b04dfeda92c67546 => generated 312 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:43:59.406897 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93bc85-6af4-47f0-a1d4-69ffc0182171 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:59.411465 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 43/88] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:43:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:43:59.417727 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54482721-1197-445a-a4a1-19287b0f7f0c None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:59.520064 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93bc85-6af4-47f0-a1d4-69ffc0182171 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:59.526605 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54482721-1197-445a-a4a1-19287b0f7f0c None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:59.564945 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98efb1be-91bb-4b42-82c5-7fdeea95619c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:59.565760 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99fae40-2ef0-485e-a942-91ea4e59737a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:59.566362 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efb1be-91bb-4b42-82c5-7fdeea95619c None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:59.566474 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fae40-2ef0-485e-a942-91ea4e59737a None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:59.566706 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fae40-2ef0-485e-a942-91ea4e59737a None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:59.567018 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efb1be-91bb-4b42-82c5-7fdeea95619c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:59.567123 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fae40-2ef0-485e-a942-91ea4e59737a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:59.567532 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efb1be-91bb-4b42-82c5-7fdeea95619c None admin] PATH_INFO: `/v3/users/f26c0bc8ce6d4932bba691c04bec3ded` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:59.567883 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99fae40-2ef0-485e-a942-91ea4e59737a None admin] RBAC: Adding query filter params (role.id=5f76e2e820b64d14b893c679427f4bed, scope.project.id=fd4139f9cd2643d3b04dfeda92c67546, user.id=f26c0bc8ce6d4932bba691c04bec3ded) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:43:59.568203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99fae40-2ef0-485e-a942-91ea4e59737a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:59.569330 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98efb1be-91bb-4b42-82c5-7fdeea95619c None admin] RBAC: Authorizing `identity:get_user(user_id=f26c0bc8ce6d4932bba691c04bec3ded)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:59.569862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99fae40-2ef0-485e-a942-91ea4e59737a None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:59.571798 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98efb1be-91bb-4b42-82c5-7fdeea95619c None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:59.573882 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 46/89] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:43:59 2022] GET /identity/v3/users/f26c0bc8ce6d4932bba691c04bec3ded => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:43:59.574825 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98efb1be-91bb-4b42-82c5-7fdeea95619c None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:59.583090 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 44/90] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:43:59 2022] GET /identity/v3/role_assignments?user.id=f26c0bc8ce6d4932bba691c04bec3ded&scope.project.id=fd4139f9cd2643d3b04dfeda92c67546&role.id=5f76e2e820b64d14b893c679427f4bed => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:43:59.584466 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99fae40-2ef0-485e-a942-91ea4e59737a None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:59.697781 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99fae40-2ef0-485e-a942-91ea4e59737a None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:59.746482 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98efb1be-91bb-4b42-82c5-7fdeea95619c None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:59.781697 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede595f4-b95f-4c2c-b958-12ac4feffe3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:59.782383 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede595f4-b95f-4c2c-b958-12ac4feffe3d None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:59.782670 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede595f4-b95f-4c2c-b958-12ac4feffe3d None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:59.782948 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede595f4-b95f-4c2c-b958-12ac4feffe3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:59.789336 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac6ed46-3650-4b2f-8074-7c981f88a84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:59.790904 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6ed46-3650-4b2f-8074-7c981f88a84f None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:59.791198 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6ed46-3650-4b2f-8074-7c981f88a84f None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:59.791478 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6ed46-3650-4b2f-8074-7c981f88a84f None admin] PATH_INFO: `/v3/users/66f1687219494ee0a402084110e075fd` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:43:59.792821 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac6ed46-3650-4b2f-8074-7c981f88a84f None admin] RBAC: Authorizing `identity:get_user(user_id=66f1687219494ee0a402084110e075fd)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:59.794832 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac6ed46-3650-4b2f-8074-7c981f88a84f None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:59.796816 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 47/91] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:43:59 2022] GET /identity/v3/users/66f1687219494ee0a402084110e075fd => generated 247 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:43:59.807556 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac6ed46-3650-4b2f-8074-7c981f88a84f None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:43:59.900865 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede595f4-b95f-4c2c-b958-12ac4feffe3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:43:59.902364 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede595f4-b95f-4c2c-b958-12ac4feffe3d None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:43:59.962469 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac6ed46-3650-4b2f-8074-7c981f88a84f None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:43:59.997317 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7226532-3d7a-4fc5-a900-d22964c2b4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:43:59.998040 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7226532-3d7a-4fc5-a900-d22964c2b4ea None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:43:59.998334 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7226532-3d7a-4fc5-a900-d22964c2b4ea None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:43:59.998620 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7226532-3d7a-4fc5-a900-d22964c2b4ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:00.044171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 45/92] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:43:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 06 14:44:00.063972 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede595f4-b95f-4c2c-b958-12ac4feffe3d None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:00.074501 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7226532-3d7a-4fc5-a900-d22964c2b4ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:00.075977 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7226532-3d7a-4fc5-a900-d22964c2b4ea None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:00.188734 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 48/93] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:43:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 06 14:44:00.190576 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7226532-3d7a-4fc5-a900-d22964c2b4ea None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:00.254769 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede595f4-b95f-4c2c-b958-12ac4feffe3d None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:00.292290 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7226532-3d7a-4fc5-a900-d22964c2b4ea None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:00.311964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fddd429-356b-4bd1-b44b-f21bbd8048b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:00.312879 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fddd429-356b-4bd1-b44b-f21bbd8048b0 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:00.313164 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fddd429-356b-4bd1-b44b-f21bbd8048b0 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:00.313441 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fddd429-356b-4bd1-b44b-f21bbd8048b0 None admin] PATH_INFO: `/v3/projects/92337f7cabcc48ff8ca007298d1c3d78` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:00.314633 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fddd429-356b-4bd1-b44b-f21bbd8048b0 None admin] RBAC: Authorizing `identity:get_project(project_id=92337f7cabcc48ff8ca007298d1c3d78)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:00.316086 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fddd429-356b-4bd1-b44b-f21bbd8048b0 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:00.317908 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 46/94] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:44:00 2022] GET /identity/v3/projects/92337f7cabcc48ff8ca007298d1c3d78 => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:00.324250 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fddd429-356b-4bd1-b44b-f21bbd8048b0 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:00.334462 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f3d48-e651-4371-bb88-933aba12975b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:00.335651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f3d48-e651-4371-bb88-933aba12975b None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:00.336086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f3d48-e651-4371-bb88-933aba12975b None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:00.336705 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f3d48-e651-4371-bb88-933aba12975b None admin] PATH_INFO: `/v3/projects/92337f7cabcc48ff8ca007298d1c3d78` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:00.338053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75f3d48-e651-4371-bb88-933aba12975b None admin] RBAC: Authorizing `identity:get_project(project_id=92337f7cabcc48ff8ca007298d1c3d78)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:00.339651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75f3d48-e651-4371-bb88-933aba12975b None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:00.342067 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 49/95] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:44:00 2022] GET /identity/v3/projects/92337f7cabcc48ff8ca007298d1c3d78 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:00.347965 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f3d48-e651-4371-bb88-933aba12975b None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:00.429470 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fddd429-356b-4bd1-b44b-f21bbd8048b0 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:00.469990 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b53ef-aa8b-44e8-bf57-51fa27b2d86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:00.470664 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b53ef-aa8b-44e8-bf57-51fa27b2d86f None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:00.470948 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b53ef-aa8b-44e8-bf57-51fa27b2d86f None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:00.471226 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b53ef-aa8b-44e8-bf57-51fa27b2d86f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:00.472063 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7b53ef-aa8b-44e8-bf57-51fa27b2d86f None admin] RBAC: Adding query filter params (role.id=d328f8040f2247d083309d839d2ff947, scope.project.id=92337f7cabcc48ff8ca007298d1c3d78, user.id=f26c0bc8ce6d4932bba691c04bec3ded) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:00.472377 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7b53ef-aa8b-44e8-bf57-51fa27b2d86f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:00.473879 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7b53ef-aa8b-44e8-bf57-51fa27b2d86f None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:00.483934 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 47/96] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:44:00 2022] GET /identity/v3/role_assignments?user.id=f26c0bc8ce6d4932bba691c04bec3ded&scope.project.id=92337f7cabcc48ff8ca007298d1c3d78&role.id=d328f8040f2247d083309d839d2ff947 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:00.515101 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f3d48-e651-4371-bb88-933aba12975b None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:00.576577 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce5fc41-5968-48cf-a6c5-a736fc2867e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:00.577622 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce5fc41-5968-48cf-a6c5-a736fc2867e5 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:00.578133 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce5fc41-5968-48cf-a6c5-a736fc2867e5 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:00.578629 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce5fc41-5968-48cf-a6c5-a736fc2867e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:00.579664 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce5fc41-5968-48cf-a6c5-a736fc2867e5 None admin] RBAC: Adding query filter params (role.id=6dcd35d1a83748759d6ef9f026077835, scope.project.id=92337f7cabcc48ff8ca007298d1c3d78, user.id=66f1687219494ee0a402084110e075fd) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:00.580239 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce5fc41-5968-48cf-a6c5-a736fc2867e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:00.586121 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce5fc41-5968-48cf-a6c5-a736fc2867e5 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:00.593910 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 50/97] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:44:00 2022] GET /identity/v3/role_assignments?user.id=66f1687219494ee0a402084110e075fd&scope.project.id=92337f7cabcc48ff8ca007298d1c3d78&role.id=6dcd35d1a83748759d6ef9f026077835 => generated 259 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:02.050386 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0c78a-87ed-4553-89ec-96eef4f74cc8 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:02.050960 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0c78a-87ed-4553-89ec-96eef4f74cc8 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:02.050960 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0c78a-87ed-4553-89ec-96eef4f74cc8 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:02.052369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 48/98] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:02.058285 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ffb6b0-828d-4f1d-b619-03b18f5e19f4 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:02.059207 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ffb6b0-828d-4f1d-b619-03b18f5e19f4 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:02.059695 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ffb6b0-828d-4f1d-b619-03b18f5e19f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:02.076527 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-b7ffb6b0-828d-4f1d-b619-03b18f5e19f4 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:02.078072 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-b7ffb6b0-828d-4f1d-b619-03b18f5e19f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:02.164451 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 51/99] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:44:02.200973 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0c78a-87ed-4553-89ec-96eef4f74cc8 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:02.331913 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0c78a-87ed-4553-89ec-96eef4f74cc8 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:02.378327 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c628d7-f481-4549-bc35-e911d4b40d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:02.379537 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c628d7-f481-4549-bc35-e911d4b40d7c None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:02.380020 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c628d7-f481-4549-bc35-e911d4b40d7c None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:02.380498 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c628d7-f481-4549-bc35-e911d4b40d7c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:02.393660 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c628d7-f481-4549-bc35-e911d4b40d7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:02.402659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-e6c628d7-f481-4549-bc35-e911d4b40d7c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 06 14:44:02.402659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:02.402659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:02.402659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:02.402659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:02.402659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:02.402659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:02.402659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:02.402659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:02.402659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:02.402659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:02.402659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:02.402659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:02.402659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:02.402659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:02.402659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:02.403569 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:02.403569 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:02.403569 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:02.403569 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:02.403569 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:02.403569 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:02.403569 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:02.403569 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:02.403569 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:02.403569 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:02.403569 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:02.403569 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:44:02.403569 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:02.403569 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:02.403569 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:02.403569 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:02.404357 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:02.404357 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:02.404357 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:02.404357 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:02.404357 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:02.404357 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:02.404357 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:02.404357 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:02.404357 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:02.404357 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:02.404357 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 06 14:44:02.404357 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:44:02.404357 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 49/100] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:44:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:02.407375 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e685a-e29b-4933-9b52-5969a99b3d70 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:02.407882 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e685a-e29b-4933-9b52-5969a99b3d70 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:02.408370 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e685a-e29b-4933-9b52-5969a99b3d70 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:02.409947 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 52/101] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:02.416285 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e685a-e29b-4933-9b52-5969a99b3d70 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:02.417148 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2d482-c980-492e-8938-d60e14535b93 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:02.417441 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2d482-c980-492e-8938-d60e14535b93 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:02.417723 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2d482-c980-492e-8938-d60e14535b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:02.441748 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-6fa2d482-c980-492e-8938-d60e14535b93 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:02.442999 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-6fa2d482-c980-492e-8938-d60e14535b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:02.539221 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 50/102] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:44:02.582179 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa2d482-c980-492e-8938-d60e14535b93 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:02.614490 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e685a-e29b-4933-9b52-5969a99b3d70 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:02.681424 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904d9fe2-2b79-4b06-9b80-f2a56dfaaffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:02.682404 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d9fe2-2b79-4b06-9b80-f2a56dfaaffb None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:02.682860 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d9fe2-2b79-4b06-9b80-f2a56dfaaffb None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:02.683278 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d9fe2-2b79-4b06-9b80-f2a56dfaaffb None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:02.684179 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904d9fe2-2b79-4b06-9b80-f2a56dfaaffb None admin] RBAC: Adding query filter params (name=reader) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:02.684644 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904d9fe2-2b79-4b06-9b80-f2a56dfaaffb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:02.686165 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904d9fe2-2b79-4b06-9b80-f2a56dfaaffb None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:02.703305 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 53/103] 10.222.0.23 () {56 vars in 1101 bytes} [Fri May 6 14:44:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:02.708716 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904d9fe2-2b79-4b06-9b80-f2a56dfaaffb None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:02.744629 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa2d482-c980-492e-8938-d60e14535b93 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:02.795178 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb80258-3506-4d1e-9781-f79b703113de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:02.796145 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb80258-3506-4d1e-9781-f79b703113de None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:02.796527 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb80258-3506-4d1e-9781-f79b703113de None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:02.796934 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb80258-3506-4d1e-9781-f79b703113de None admin] PATH_INFO: `/v3/domains/default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:02.798201 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb80258-3506-4d1e-9781-f79b703113de None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:02.804628 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb80258-3506-4d1e-9781-f79b703113de None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:02.804628 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 51/104] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:44:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:02.808015 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb80258-3506-4d1e-9781-f79b703113de None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:02.914960 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904d9fe2-2b79-4b06-9b80-f2a56dfaaffb None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:02.962881 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb80258-3506-4d1e-9781-f79b703113de None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:03.002215 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee09bf-1b05-4a02-8948-e8a2385bc87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:03.003108 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee09bf-1b05-4a02-8948-e8a2385bc87f None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:03.003391 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee09bf-1b05-4a02-8948-e8a2385bc87f None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:03.003669 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee09bf-1b05-4a02-8948-e8a2385bc87f None admin] PATH_INFO: `/v3/users/cdcebaa21c554387a0c7293ad5033b28` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:03.039100 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ee09bf-1b05-4a02-8948-e8a2385bc87f None admin] RBAC: Authorizing `identity:get_user(user_id=cdcebaa21c554387a0c7293ad5033b28)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:03.041303 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa70606-f7fe-45ab-9dd2-76cb0cba182f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:03.041984 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa70606-f7fe-45ab-9dd2-76cb0cba182f None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:03.042275 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa70606-f7fe-45ab-9dd2-76cb0cba182f None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:03.042558 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa70606-f7fe-45ab-9dd2-76cb0cba182f None admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:03.043012 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ee09bf-1b05-4a02-8948-e8a2385bc87f None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:03.043291 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa70606-f7fe-45ab-9dd2-76cb0cba182f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:03.048640 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa70606-f7fe-45ab-9dd2-76cb0cba182f None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:03.049740 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 54/105] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:44:02 2022] GET /identity/v3/users/cdcebaa21c554387a0c7293ad5033b28 => generated 289 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:03.055969 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee09bf-1b05-4a02-8948-e8a2385bc87f None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:03.087294 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 52/106] 10.222.0.23 () {60 vars in 1141 bytes} [Fri May 6 14:44:02 2022] POST /identity/v3/projects => generated 302 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:44:03.158627 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec019354-bcbe-4ca6-a264-1311b3f6ef33 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:03.159299 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec019354-bcbe-4ca6-a264-1311b3f6ef33 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:03.159835 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec019354-bcbe-4ca6-a264-1311b3f6ef33 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:03.163937 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 53/107] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:44:03.172712 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7615cb-4b3d-437d-a0a5-d186a031987b None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:03.173001 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7615cb-4b3d-437d-a0a5-d186a031987b None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:03.173370 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7615cb-4b3d-437d-a0a5-d186a031987b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:03.201246 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-cf7615cb-4b3d-437d-a0a5-d186a031987b None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:03.202470 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-cf7615cb-4b3d-437d-a0a5-d186a031987b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:03.235244 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee09bf-1b05-4a02-8948-e8a2385bc87f None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:03.275918 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608013d8-a7bb-47f9-8ca7-abbad0a057fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:03.276799 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608013d8-a7bb-47f9-8ca7-abbad0a057fc None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:03.277090 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608013d8-a7bb-47f9-8ca7-abbad0a057fc None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:03.277370 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608013d8-a7bb-47f9-8ca7-abbad0a057fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:03.360513 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 54/108] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:44:03.361996 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5128f87-ffa8-424f-bf68-9d7d8d95c7af None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:03.362561 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5128f87-ffa8-424f-bf68-9d7d8d95c7af None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:03.363036 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5128f87-ffa8-424f-bf68-9d7d8d95c7af None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:03.371015 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 55/109] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:03 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:03.376525 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773c946-5559-4884-b6c6-bcb643e4a754 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:03.377067 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773c946-5559-4884-b6c6-bcb643e4a754 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:03.377557 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773c946-5559-4884-b6c6-bcb643e4a754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:03.386706 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608013d8-a7bb-47f9-8ca7-abbad0a057fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:03.388419 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608013d8-a7bb-47f9-8ca7-abbad0a057fc None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:03.401567 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-7773c946-5559-4884-b6c6-bcb643e4a754 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:03.403063 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-7773c946-5559-4884-b6c6-bcb643e4a754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:03.479544 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 56/110] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 06 14:44:03.480751 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7773c946-5559-4884-b6c6-bcb643e4a754 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:03.488393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 55/111] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:44:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 06 14:44:03.503263 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608013d8-a7bb-47f9-8ca7-abbad0a057fc None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:03.594373 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7773c946-5559-4884-b6c6-bcb643e4a754 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:03.627649 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608013d8-a7bb-47f9-8ca7-abbad0a057fc None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:03.630336 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d163eb1-fd66-4d82-9ee7-9d9d59baec1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:03.631140 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d163eb1-fd66-4d82-9ee7-9d9d59baec1e None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:03.631422 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d163eb1-fd66-4d82-9ee7-9d9d59baec1e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:03.631700 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d163eb1-fd66-4d82-9ee7-9d9d59baec1e None admin] PATH_INFO: `/v3/roles/member` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:03.642411 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d163eb1-fd66-4d82-9ee7-9d9d59baec1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:03.651279 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-0d163eb1-fd66-4d82-9ee7-9d9d59baec1e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 06 14:44:03.651279 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:03.651279 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:03.651279 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:03.651279 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:03.651279 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:03.651279 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:03.651279 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:03.651279 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:03.651279 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:03.651279 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:03.651279 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:03.651279 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:03.651279 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:03.651279 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:03.651279 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:03.652362 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:03.652362 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:03.652362 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:03.652362 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:03.652362 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:03.652362 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:03.652362 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:03.652362 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:03.652362 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:03.652362 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:03.652362 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:03.652362 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:44:03.652362 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:03.652362 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:03.652362 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:03.652362 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:03.653365 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:03.653365 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:03.653365 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:03.653365 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:03.653365 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:03.653365 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:03.653365 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:03.653365 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:03.653365 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:03.653365 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:03.653365 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 06 14:44:03.653365 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:44:03.655464 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 57/112] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:44:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:03.656745 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d163eb1-fd66-4d82-9ee7-9d9d59baec1e None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:03.672256 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d13314b-1ce5-4a32-af88-9177e7f0a389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:03.673619 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d13314b-1ce5-4a32-af88-9177e7f0a389 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:03.674470 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d13314b-1ce5-4a32-af88-9177e7f0a389 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:03.675231 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d13314b-1ce5-4a32-af88-9177e7f0a389 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:03.689075 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d13314b-1ce5-4a32-af88-9177e7f0a389 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:03.699400 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-7d13314b-1ce5-4a32-af88-9177e7f0a389 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 06 14:44:03.699400 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:03.699400 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:03.699400 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:03.699400 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:03.699400 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:03.699400 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:03.699400 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:03.699400 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:03.699400 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:03.699400 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:03.699400 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:03.699400 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:03.699400 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:03.699400 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:03.699400 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:03.700292 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:03.700292 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:03.700292 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:03.700292 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:03.700292 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:03.700292 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:03.700292 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:03.700292 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:03.700292 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:03.700292 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:03.700292 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:03.700292 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:44:03.700292 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:03.700292 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:03.700292 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:03.700292 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:03.701080 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:03.701080 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:03.701080 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:03.701080 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:03.701080 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:03.701080 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:03.701080 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:03.701080 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:03.701080 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:03.701080 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:03.701080 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 06 14:44:03.701080 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:44:03.707913 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 56/113] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:44:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:44:03.707913 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6f381-b5c9-4af2-88ff-bf8fa896c5ae None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:03.707913 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6f381-b5c9-4af2-88ff-bf8fa896c5ae None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:03.707913 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6f381-b5c9-4af2-88ff-bf8fa896c5ae None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:03.707913 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 57/114] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:03.707913 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e6f381-b5c9-4af2-88ff-bf8fa896c5ae None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:03.793405 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d163eb1-fd66-4d82-9ee7-9d9d59baec1e None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:03.823211 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e6f381-b5c9-4af2-88ff-bf8fa896c5ae None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:03.839948 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848df5ed-a15f-404a-950a-ed8c6f85316f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:03.841240 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848df5ed-a15f-404a-950a-ed8c6f85316f None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:03.841982 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848df5ed-a15f-404a-950a-ed8c6f85316f None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:03.842398 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848df5ed-a15f-404a-950a-ed8c6f85316f None admin] PATH_INFO: `/v3/projects/92337f7cabcc48ff8ca007298d1c3d78` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:03.843732 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848df5ed-a15f-404a-950a-ed8c6f85316f None admin] RBAC: Authorizing `identity:get_project(project_id=92337f7cabcc48ff8ca007298d1c3d78)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:03.846171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848df5ed-a15f-404a-950a-ed8c6f85316f None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:03.848867 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 58/115] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:44:03 2022] GET /identity/v3/projects/92337f7cabcc48ff8ca007298d1c3d78 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:03.850465 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a5917-e2b7-47a6-be92-e90c6f9152ef None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:03.850954 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a5917-e2b7-47a6-be92-e90c6f9152ef None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:03.851378 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a5917-e2b7-47a6-be92-e90c6f9152ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:03.863493 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133103fb-b4c8-41c3-9086-2e0d9fe99431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:03.864261 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133103fb-b4c8-41c3-9086-2e0d9fe99431 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:03.864552 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133103fb-b4c8-41c3-9086-2e0d9fe99431 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:03.864876 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133103fb-b4c8-41c3-9086-2e0d9fe99431 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:03.865627 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133103fb-b4c8-41c3-9086-2e0d9fe99431 None admin] RBAC: Adding query filter params (name=member) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:03.865936 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133103fb-b4c8-41c3-9086-2e0d9fe99431 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:03.867327 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133103fb-b4c8-41c3-9086-2e0d9fe99431 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:03.873944 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-2e4a5917-e2b7-47a6-be92-e90c6f9152ef None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:03.875348 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-2e4a5917-e2b7-47a6-be92-e90c6f9152ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:03.875628 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 58/116] 10.222.0.23 () {56 vars in 1101 bytes} [Fri May 6 14:44:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:03.876448 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133103fb-b4c8-41c3-9086-2e0d9fe99431 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:03.950564 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 59/117] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:44:03.951765 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4a5917-e2b7-47a6-be92-e90c6f9152ef None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:03.982409 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133103fb-b4c8-41c3-9086-2e0d9fe99431 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:04.023507 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47864a-b067-4929-9a9e-8995f4e252fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:04.024792 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47864a-b067-4929-9a9e-8995f4e252fc None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:04.025345 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47864a-b067-4929-9a9e-8995f4e252fc None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:04.025345 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47864a-b067-4929-9a9e-8995f4e252fc None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:04.026069 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da47864a-b067-4929-9a9e-8995f4e252fc None admin] RBAC: Adding query filter params (name=member) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:04.026380 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da47864a-b067-4929-9a9e-8995f4e252fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:04.028469 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da47864a-b067-4929-9a9e-8995f4e252fc None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:04.036373 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 59/118] 10.222.0.23 () {56 vars in 1101 bytes} [Fri May 6 14:44:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:04.037252 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47864a-b067-4929-9a9e-8995f4e252fc None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:04.061682 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4a5917-e2b7-47a6-be92-e90c6f9152ef None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:04.100488 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0831686b-265b-4eb8-b544-73b8d82c7d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:04.102351 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0831686b-265b-4eb8-b544-73b8d82c7d33 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:04.103161 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0831686b-265b-4eb8-b544-73b8d82c7d33 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:04.103658 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0831686b-265b-4eb8-b544-73b8d82c7d33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:04.104821 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0831686b-265b-4eb8-b544-73b8d82c7d33 None admin] RBAC: Adding query filter params (role.id=e5c0610b69d640e595dccba09f44b81e, scope.project.id=92337f7cabcc48ff8ca007298d1c3d78, user.id=cdcebaa21c554387a0c7293ad5033b28) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:04.105411 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0831686b-265b-4eb8-b544-73b8d82c7d33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:04.107132 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0831686b-265b-4eb8-b544-73b8d82c7d33 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:04.114095 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 60/119] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:44:03 2022] GET /identity/v3/role_assignments?user.id=cdcebaa21c554387a0c7293ad5033b28&scope.project.id=92337f7cabcc48ff8ca007298d1c3d78&role.id=e5c0610b69d640e595dccba09f44b81e => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:04.115332 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0831686b-265b-4eb8-b544-73b8d82c7d33 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:04.167383 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47864a-b067-4929-9a9e-8995f4e252fc None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:04.234412 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f48c862-1f5d-4b75-9552-4649d1d86df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:04.235294 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48c862-1f5d-4b75-9552-4649d1d86df8 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:04.235858 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48c862-1f5d-4b75-9552-4649d1d86df8 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:04.235858 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48c862-1f5d-4b75-9552-4649d1d86df8 None admin] PATH_INFO: `/v3/users/f26c0bc8ce6d4932bba691c04bec3ded` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:04.239739 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0831686b-265b-4eb8-b544-73b8d82c7d33 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:04.241497 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f48c862-1f5d-4b75-9552-4649d1d86df8 None admin] RBAC: Authorizing `identity:get_user(user_id=f26c0bc8ce6d4932bba691c04bec3ded)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:04.244656 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f48c862-1f5d-4b75-9552-4649d1d86df8 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:04.245634 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 60/120] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:44:04 2022] GET /identity/v3/users/f26c0bc8ce6d4932bba691c04bec3ded => generated 275 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:04.246430 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f48c862-1f5d-4b75-9552-4649d1d86df8 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:04.279638 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ad3481-0bef-4428-934f-7e1ecdae0a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:04.281208 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad3481-0bef-4428-934f-7e1ecdae0a05 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:04.281708 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad3481-0bef-4428-934f-7e1ecdae0a05 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:04.282204 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad3481-0bef-4428-934f-7e1ecdae0a05 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:04.296579 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ad3481-0bef-4428-934f-7e1ecdae0a05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:04.309707 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-36ad3481-0bef-4428-934f-7e1ecdae0a05 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 06 14:44:04.309707 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:04.309707 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:04.309707 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:04.309707 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:04.309707 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:04.309707 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:04.309707 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:04.309707 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:04.309707 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:04.309707 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:04.309707 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:04.309707 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:04.309707 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:04.309707 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:04.309707 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:04.310634 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:04.310634 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:04.310634 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:04.310634 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:04.310634 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:04.310634 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:04.310634 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:04.310634 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:04.310634 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:04.310634 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:04.310634 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:04.310634 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:44:04.310634 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:04.310634 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:04.310634 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:04.310634 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:04.311414 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:04.311414 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:04.311414 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:04.311414 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:04.311414 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:04.311414 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:04.311414 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:04.311414 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:04.311414 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:04.311414 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:04.311414 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 06 14:44:04.311414 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:44:04.314645 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 61/121] 10.222.0.23 () {56 vars in 1114 bytes} [Fri May 6 14:44:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:04.316232 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ad3481-0bef-4428-934f-7e1ecdae0a05 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:04.377565 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f48c862-1f5d-4b75-9552-4649d1d86df8 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:04.429228 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d13c33c-40d8-4258-82d8-1e8775d72e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:04.430116 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d13c33c-40d8-4258-82d8-1e8775d72e70 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:04.430405 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d13c33c-40d8-4258-82d8-1e8775d72e70 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:04.430687 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d13c33c-40d8-4258-82d8-1e8775d72e70 None admin] PATH_INFO: `/v3/users/f26c0bc8ce6d4932bba691c04bec3ded` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:04.432242 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d13c33c-40d8-4258-82d8-1e8775d72e70 None admin] RBAC: Authorizing `identity:get_user(user_id=f26c0bc8ce6d4932bba691c04bec3ded)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:04.433888 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d13c33c-40d8-4258-82d8-1e8775d72e70 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:04.435532 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 61/122] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:44:04 2022] GET /identity/v3/users/f26c0bc8ce6d4932bba691c04bec3ded => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:04.436765 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a9af9-b24f-4981-95d7-20314f8a0b34 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:04.437055 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a9af9-b24f-4981-95d7-20314f8a0b34 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:04.437338 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a9af9-b24f-4981-95d7-20314f8a0b34 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:04.438550 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 62/123] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:04.439244 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a9af9-b24f-4981-95d7-20314f8a0b34 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:04.447828 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ad3481-0bef-4428-934f-7e1ecdae0a05 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:04.482989 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059a0ae5-437a-4fa1-bd29-c43fce5a8447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:04.484196 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a0ae5-437a-4fa1-bd29-c43fce5a8447 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:04.484745 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a0ae5-437a-4fa1-bd29-c43fce5a8447 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:04.485229 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a0ae5-437a-4fa1-bd29-c43fce5a8447 None admin] PATH_INFO: `/v3/projects/fd4139f9cd2643d3b04dfeda92c67546` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:04.486607 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059a0ae5-437a-4fa1-bd29-c43fce5a8447 None admin] RBAC: Authorizing `identity:get_project(project_id=fd4139f9cd2643d3b04dfeda92c67546)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:04.488243 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059a0ae5-437a-4fa1-bd29-c43fce5a8447 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:04.490157 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 62/124] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:44:04 2022] GET /identity/v3/projects/fd4139f9cd2643d3b04dfeda92c67546 => generated 312 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:04.491179 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059a0ae5-437a-4fa1-bd29-c43fce5a8447 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:04.550958 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a9af9-b24f-4981-95d7-20314f8a0b34 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:04.585536 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1235f5ec-b77a-49ec-bbee-6f701ecc5f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:04.586140 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235f5ec-b77a-49ec-bbee-6f701ecc5f6a None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:04.586462 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235f5ec-b77a-49ec-bbee-6f701ecc5f6a None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:04.586739 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235f5ec-b77a-49ec-bbee-6f701ecc5f6a None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:04.587543 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1235f5ec-b77a-49ec-bbee-6f701ecc5f6a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:04.587791 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1235f5ec-b77a-49ec-bbee-6f701ecc5f6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:04.589224 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1235f5ec-b77a-49ec-bbee-6f701ecc5f6a None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:04.598455 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 63/125] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:44:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:04.599572 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644cfdc-f91f-4f25-953e-55824d09d9e7 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:04.599867 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644cfdc-f91f-4f25-953e-55824d09d9e7 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:04.600146 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644cfdc-f91f-4f25-953e-55824d09d9e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:04.600912 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059a0ae5-437a-4fa1-bd29-c43fce5a8447 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:04.619094 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-3644cfdc-f91f-4f25-953e-55824d09d9e7 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:04.620201 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-3644cfdc-f91f-4f25-953e-55824d09d9e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:04.639976 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c86a19-905e-4cca-b4f3-a9589596a2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:04.641238 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c86a19-905e-4cca-b4f3-a9589596a2f3 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:04.641728 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c86a19-905e-4cca-b4f3-a9589596a2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:04.642191 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c86a19-905e-4cca-b4f3-a9589596a2f3 None admin] PATH_INFO: `/v3/projects/92337f7cabcc48ff8ca007298d1c3d78` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:04.643560 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c86a19-905e-4cca-b4f3-a9589596a2f3 None admin] RBAC: Authorizing `identity:get_project(project_id=92337f7cabcc48ff8ca007298d1c3d78)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:04.645221 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c86a19-905e-4cca-b4f3-a9589596a2f3 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:04.647121 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 63/126] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:44:04 2022] GET /identity/v3/projects/92337f7cabcc48ff8ca007298d1c3d78 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:04.648117 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c86a19-905e-4cca-b4f3-a9589596a2f3 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:04.690594 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 64/127] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:44:04.691292 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3644cfdc-f91f-4f25-953e-55824d09d9e7 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:04.752995 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c86a19-905e-4cca-b4f3-a9589596a2f3 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:04.788165 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd293465-35ca-4462-91a8-4cfec8ce066b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:04.789329 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd293465-35ca-4462-91a8-4cfec8ce066b None admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:04.790349 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd293465-35ca-4462-91a8-4cfec8ce066b None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:04.790770 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd293465-35ca-4462-91a8-4cfec8ce066b None admin] PATH_INFO: `/v3/projects/fd4139f9cd2643d3b04dfeda92c67546/users/f26c0bc8ce6d4932bba691c04bec3ded/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:04.793005 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd293465-35ca-4462-91a8-4cfec8ce066b None admin] RBAC: Authorizing `identity:create_grant(project_id=fd4139f9cd2643d3b04dfeda92c67546, user_id=f26c0bc8ce6d4932bba691c04bec3ded, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:04.794713 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd293465-35ca-4462-91a8-4cfec8ce066b None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:04.799147 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3644cfdc-f91f-4f25-953e-55824d09d9e7 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:04.815322 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 64/128] 10.222.0.23 () {58 vars in 1439 bytes} [Fri May 6 14:44:04 2022] PUT /identity/v3/projects/fd4139f9cd2643d3b04dfeda92c67546/users/f26c0bc8ce6d4932bba691c04bec3ded/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:44:04.816849 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd293465-35ca-4462-91a8-4cfec8ce066b None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:04.840584 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af3add-a42e-4809-aee7-c0b12c9ed5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:04.841883 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af3add-a42e-4809-aee7-c0b12c9ed5c3 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:04.842249 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af3add-a42e-4809-aee7-c0b12c9ed5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:04.842603 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af3add-a42e-4809-aee7-c0b12c9ed5c3 None admin] PATH_INFO: `/v3/users/f26c0bc8ce6d4932bba691c04bec3ded` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:04.844279 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af3add-a42e-4809-aee7-c0b12c9ed5c3 None admin] RBAC: Authorizing `identity:get_user(user_id=f26c0bc8ce6d4932bba691c04bec3ded)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:04.846306 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af3add-a42e-4809-aee7-c0b12c9ed5c3 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:04.848142 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 65/129] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:44:04 2022] GET /identity/v3/users/f26c0bc8ce6d4932bba691c04bec3ded => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:04.849648 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af3add-a42e-4809-aee7-c0b12c9ed5c3 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:04.970971 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd293465-35ca-4462-91a8-4cfec8ce066b None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:05.021618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af3add-a42e-4809-aee7-c0b12c9ed5c3 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:05.044063 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ba90f-465c-45b1-a119-d2e2ef7920b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:05.044996 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ba90f-465c-45b1-a119-d2e2ef7920b1 None admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:05.045283 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ba90f-465c-45b1-a119-d2e2ef7920b1 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:05.045560 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ba90f-465c-45b1-a119-d2e2ef7920b1 None admin] PATH_INFO: `/v3/projects/92337f7cabcc48ff8ca007298d1c3d78/users/f26c0bc8ce6d4932bba691c04bec3ded/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:05.049510 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2ba90f-465c-45b1-a119-d2e2ef7920b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=92337f7cabcc48ff8ca007298d1c3d78, user_id=f26c0bc8ce6d4932bba691c04bec3ded, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:05.050994 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2ba90f-465c-45b1-a119-d2e2ef7920b1 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:05.070464 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 65/130] 10.222.0.23 () {58 vars in 1439 bytes} [Fri May 6 14:44:04 2022] PUT /identity/v3/projects/92337f7cabcc48ff8ca007298d1c3d78/users/f26c0bc8ce6d4932bba691c04bec3ded/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:44:05.071990 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ba90f-465c-45b1-a119-d2e2ef7920b1 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:05.106696 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def76bb-fade-4037-b7ab-4b67b359385d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:05.107446 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def76bb-fade-4037-b7ab-4b67b359385d None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:05.107739 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def76bb-fade-4037-b7ab-4b67b359385d None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:05.108066 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def76bb-fade-4037-b7ab-4b67b359385d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:05.149714 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0def76bb-fade-4037-b7ab-4b67b359385d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:05.157766 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-0def76bb-fade-4037-b7ab-4b67b359385d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:44:05.157766 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:05.157766 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:05.157766 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:05.157766 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:05.157766 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:05.157766 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:05.157766 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:05.157766 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:05.157766 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:05.157766 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:05.157766 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:05.157766 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:05.157766 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:05.157766 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:05.157766 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:05.158739 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:05.158739 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:05.158739 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:05.158739 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:05.158739 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:05.158739 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:05.158739 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:05.158739 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:05.158739 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:05.158739 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:05.158739 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:05.158739 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:44:05.158739 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:05.158739 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:05.158739 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:05.158739 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:05.159648 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:05.159648 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:05.159648 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:05.159648 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:05.159648 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:05.159648 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:05.159648 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:05.159648 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:05.159648 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:05.159648 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:05.159648 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:44:05.159648 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:44:05.159648 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 66/131] 10.222.0.23 () {56 vars in 1096 bytes} [Fri May 6 14:44:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:05.177693 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def76bb-fade-4037-b7ab-4b67b359385d None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:05.220292 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ba90f-465c-45b1-a119-d2e2ef7920b1 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:05.285907 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852e583-6944-4471-be5e-88b47ca93ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:05.286796 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852e583-6944-4471-be5e-88b47ca93ddc None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:05.287082 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852e583-6944-4471-be5e-88b47ca93ddc None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:05.287682 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852e583-6944-4471-be5e-88b47ca93ddc None admin] PATH_INFO: `/v3/projects/92337f7cabcc48ff8ca007298d1c3d78` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:05.293677 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d852e583-6944-4471-be5e-88b47ca93ddc None admin] RBAC: Authorizing `identity:get_project(project_id=92337f7cabcc48ff8ca007298d1c3d78)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:05.295304 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d852e583-6944-4471-be5e-88b47ca93ddc None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:05.297179 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 66/132] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:44:05 2022] GET /identity/v3/projects/92337f7cabcc48ff8ca007298d1c3d78 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:05.304366 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852e583-6944-4471-be5e-88b47ca93ddc None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:05.409631 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852e583-6944-4471-be5e-88b47ca93ddc None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:05.420532 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def76bb-fade-4037-b7ab-4b67b359385d None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:05.444615 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88682836-e01b-4a15-80ca-c9e9a4137261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:05.445529 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88682836-e01b-4a15-80ca-c9e9a4137261 None admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:05.446453 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88682836-e01b-4a15-80ca-c9e9a4137261 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:05.446730 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88682836-e01b-4a15-80ca-c9e9a4137261 None admin] PATH_INFO: `/v3/projects/92337f7cabcc48ff8ca007298d1c3d78/users/f26c0bc8ce6d4932bba691c04bec3ded/roles/d328f8040f2247d083309d839d2ff947` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:05.453448 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88682836-e01b-4a15-80ca-c9e9a4137261 None admin] RBAC: Authorizing `identity:create_grant(project_id=92337f7cabcc48ff8ca007298d1c3d78, user_id=f26c0bc8ce6d4932bba691c04bec3ded, role_id=d328f8040f2247d083309d839d2ff947)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:05.454915 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88682836-e01b-4a15-80ca-c9e9a4137261 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:05.474471 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 67/133] 10.222.0.23 () {58 vars in 1439 bytes} [Fri May 6 14:44:05 2022] PUT /identity/v3/projects/92337f7cabcc48ff8ca007298d1c3d78/users/f26c0bc8ce6d4932bba691c04bec3ded/roles/d328f8040f2247d083309d839d2ff947 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 14:44:05.483365 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dccbbed-e20b-4aaa-8154-27220671088b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:05.484061 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dccbbed-e20b-4aaa-8154-27220671088b None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:05.484348 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dccbbed-e20b-4aaa-8154-27220671088b None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:05.484654 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dccbbed-e20b-4aaa-8154-27220671088b None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:05.485424 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dccbbed-e20b-4aaa-8154-27220671088b None admin] RBAC: Adding query filter params (name=admin) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:05.485740 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dccbbed-e20b-4aaa-8154-27220671088b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:05.487207 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dccbbed-e20b-4aaa-8154-27220671088b None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:05.499942 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 67/134] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:44:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:05.506801 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88682836-e01b-4a15-80ca-c9e9a4137261 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:05.621363 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88682836-e01b-4a15-80ca-c9e9a4137261 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:05.657259 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a98bbc0-fc85-4ca2-a4fa-5fef7d3fcd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:05.658171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98bbc0-fc85-4ca2-a4fa-5fef7d3fcd69 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:05.658460 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98bbc0-fc85-4ca2-a4fa-5fef7d3fcd69 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:05.658743 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98bbc0-fc85-4ca2-a4fa-5fef7d3fcd69 None admin] PATH_INFO: `/v3/users/66f1687219494ee0a402084110e075fd` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:05.660094 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a98bbc0-fc85-4ca2-a4fa-5fef7d3fcd69 None admin] RBAC: Authorizing `identity:get_user(user_id=66f1687219494ee0a402084110e075fd)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:05.661549 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a98bbc0-fc85-4ca2-a4fa-5fef7d3fcd69 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:05.663220 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 68/135] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:44:05 2022] GET /identity/v3/users/66f1687219494ee0a402084110e075fd => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:05.669462 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dccbbed-e20b-4aaa-8154-27220671088b None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:05.777026 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dccbbed-e20b-4aaa-8154-27220671088b None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:05.823121 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6a167-1c14-44e2-8842-a85129f704b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:05.824014 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6a167-1c14-44e2-8842-a85129f704b1 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:05.824308 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6a167-1c14-44e2-8842-a85129f704b1 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:05.824591 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6a167-1c14-44e2-8842-a85129f704b1 None admin] PATH_INFO: `/v3/projects/92337f7cabcc48ff8ca007298d1c3d78` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:05.825949 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e6a167-1c14-44e2-8842-a85129f704b1 None admin] RBAC: Authorizing `identity:get_project(project_id=92337f7cabcc48ff8ca007298d1c3d78)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:05.827516 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e6a167-1c14-44e2-8842-a85129f704b1 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:05.829818 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 68/136] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:44:05 2022] GET /identity/v3/projects/92337f7cabcc48ff8ca007298d1c3d78 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:05.835816 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a98bbc0-fc85-4ca2-a4fa-5fef7d3fcd69 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:05.953671 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a98bbc0-fc85-4ca2-a4fa-5fef7d3fcd69 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:05.993551 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5385c334-8f8e-4f4d-b3d8-1fc41d1e6db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:05.994422 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385c334-8f8e-4f4d-b3d8-1fc41d1e6db3 None admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:05.994708 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385c334-8f8e-4f4d-b3d8-1fc41d1e6db3 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:05.994990 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385c334-8f8e-4f4d-b3d8-1fc41d1e6db3 None admin] PATH_INFO: `/v3/projects/92337f7cabcc48ff8ca007298d1c3d78/users/66f1687219494ee0a402084110e075fd/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:06.001610 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5385c334-8f8e-4f4d-b3d8-1fc41d1e6db3 None admin] RBAC: Authorizing `identity:create_grant(project_id=92337f7cabcc48ff8ca007298d1c3d78, user_id=66f1687219494ee0a402084110e075fd, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:06.001610 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5385c334-8f8e-4f4d-b3d8-1fc41d1e6db3 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:06.023346 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 69/137] 10.222.0.23 () {58 vars in 1439 bytes} [Fri May 6 14:44:05 2022] PUT /identity/v3/projects/92337f7cabcc48ff8ca007298d1c3d78/users/66f1687219494ee0a402084110e075fd/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:44:06.082208 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c4f80-11c9-4696-aef1-d15f3bab8c45 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:06.082542 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c4f80-11c9-4696-aef1-d15f3bab8c45 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:06.082784 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c4f80-11c9-4696-aef1-d15f3bab8c45 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:06.084284 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 69/138] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:06.090421 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296e648-1159-4b24-acf5-5ad7b6872a8a None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:06.091000 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296e648-1159-4b24-acf5-5ad7b6872a8a None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:06.091502 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296e648-1159-4b24-acf5-5ad7b6872a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:06.115150 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-2296e648-1159-4b24-acf5-5ad7b6872a8a None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:06.116769 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-2296e648-1159-4b24-acf5-5ad7b6872a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:06.211255 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 70/139] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:44:06.248527 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c4f80-11c9-4696-aef1-d15f3bab8c45 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:06.399792 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c4f80-11c9-4696-aef1-d15f3bab8c45 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:06.444101 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f8862-220c-46c7-a2a0-3f2de382897c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:06.444937 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f8862-220c-46c7-a2a0-3f2de382897c None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:06.445224 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f8862-220c-46c7-a2a0-3f2de382897c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:06.445506 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f8862-220c-46c7-a2a0-3f2de382897c None admin] PATH_INFO: `/v3/domains/default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:06.453406 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f8862-220c-46c7-a2a0-3f2de382897c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:06.454880 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f8862-220c-46c7-a2a0-3f2de382897c None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:06.456576 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 70/140] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:44:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:06.462892 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296e648-1159-4b24-acf5-5ad7b6872a8a None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:06.599309 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296e648-1159-4b24-acf5-5ad7b6872a8a None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:06.636219 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f50b67-f616-4c60-bb73-ba920aab59a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:06.636937 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f50b67-f616-4c60-bb73-ba920aab59a6 None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:06.637223 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f50b67-f616-4c60-bb73-ba920aab59a6 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:06.637504 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f50b67-f616-4c60-bb73-ba920aab59a6 None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:06.638214 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f50b67-f616-4c60-bb73-ba920aab59a6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:06.639675 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f50b67-f616-4c60-bb73-ba920aab59a6 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:06.660495 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 71/141] 10.222.0.23 () {60 vars in 1132 bytes} [Fri May 6 14:44:06 2022] POST /identity/v3/users => generated 283 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:44:07.350716 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b862c25-2433-42a6-b5ee-df97eea5b59a None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:07.351221 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b862c25-2433-42a6-b5ee-df97eea5b59a None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:07.351624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b862c25-2433-42a6-b5ee-df97eea5b59a None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:07.353730 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 71/142] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:07.361586 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0056d-30ac-46dd-8619-a760f7473d45 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:07.361891 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0056d-30ac-46dd-8619-a760f7473d45 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:07.362175 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0056d-30ac-46dd-8619-a760f7473d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:07.382670 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-5db0056d-30ac-46dd-8619-a760f7473d45 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:07.384530 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-5db0056d-30ac-46dd-8619-a760f7473d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:07.459954 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 72/143] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:44:07.485482 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b862c25-2433-42a6-b5ee-df97eea5b59a None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:07.594630 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b862c25-2433-42a6-b5ee-df97eea5b59a None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:07.629495 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e9af0d-c1e2-4f95-930a-c658a8879487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:07.630490 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9af0d-c1e2-4f95-930a-c658a8879487 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:07.630784 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9af0d-c1e2-4f95-930a-c658a8879487 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:07.631064 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9af0d-c1e2-4f95-930a-c658a8879487 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:07.642854 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e9af0d-c1e2-4f95-930a-c658a8879487 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:07.651026 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-32e9af0d-c1e2-4f95-930a-c658a8879487 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 06 14:44:07.651026 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:07.651026 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:07.651026 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:07.651026 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:07.651026 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:07.651026 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:07.651026 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:07.651026 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:07.651026 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:07.651026 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:07.651026 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:07.651026 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:07.651026 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:07.651026 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:07.651026 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:07.653559 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:07.653559 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:07.653559 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:07.653559 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:07.653559 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:07.653559 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:07.653559 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:07.653559 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:07.653559 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:07.653559 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:07.653559 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:07.653559 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:44:07.653559 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:07.653559 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:07.653559 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:07.653559 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:07.655420 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:07.655420 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:07.655420 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:07.655420 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:07.655420 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:07.655420 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:07.655420 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:07.655420 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:07.655420 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:07.655420 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:07.655420 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 06 14:44:07.655420 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:44:07.655420 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 72/144] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:44:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:07.664623 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db0056d-30ac-46dd-8619-a760f7473d45 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:07.765303 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db0056d-30ac-46dd-8619-a760f7473d45 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:07.826013 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c81596c-240f-438a-a355-3b96c2761713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:07.827072 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c81596c-240f-438a-a355-3b96c2761713 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:07.827637 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c81596c-240f-438a-a355-3b96c2761713 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:07.828088 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c81596c-240f-438a-a355-3b96c2761713 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:07.829272 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c81596c-240f-438a-a355-3b96c2761713 None admin] RBAC: Adding query filter params (name=reader) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:07.829804 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c81596c-240f-438a-a355-3b96c2761713 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:07.832540 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c81596c-240f-438a-a355-3b96c2761713 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:07.846993 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 73/145] 10.222.0.23 () {56 vars in 1101 bytes} [Fri May 6 14:44:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:07.855432 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e9af0d-c1e2-4f95-930a-c658a8879487 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:08.006821 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96086a4-d502-41d1-b55e-433c4b0fe6f7 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:08.007283 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96086a4-d502-41d1-b55e-433c4b0fe6f7 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:08.007759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96086a4-d502-41d1-b55e-433c4b0fe6f7 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:08.010824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 74/146] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:08.020699 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798c782-f5d4-4b96-a77a-3526ddb2b90d None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:08.021201 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798c782-f5d4-4b96-a77a-3526ddb2b90d None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:08.021755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798c782-f5d4-4b96-a77a-3526ddb2b90d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:08.040962 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e9af0d-c1e2-4f95-930a-c658a8879487 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:08.056477 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-b798c782-f5d4-4b96-a77a-3526ddb2b90d None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:08.059362 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b798c782-f5d4-4b96-a77a-3526ddb2b90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:08.098908 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59891aa4-efbc-4223-a1b9-827c60bb35a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:08.099794 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59891aa4-efbc-4223-a1b9-827c60bb35a3 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:08.100080 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59891aa4-efbc-4223-a1b9-827c60bb35a3 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:08.100356 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59891aa4-efbc-4223-a1b9-827c60bb35a3 None admin] PATH_INFO: `/v3/users/cdcebaa21c554387a0c7293ad5033b28` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:08.103357 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59891aa4-efbc-4223-a1b9-827c60bb35a3 None admin] RBAC: Authorizing `identity:get_user(user_id=cdcebaa21c554387a0c7293ad5033b28)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:08.104459 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59891aa4-efbc-4223-a1b9-827c60bb35a3 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:08.109639 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 73/147] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:44:07 2022] GET /identity/v3/users/cdcebaa21c554387a0c7293ad5033b28 => generated 289 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:08.118049 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59891aa4-efbc-4223-a1b9-827c60bb35a3 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:08.200513 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 75/148] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:44:08.257286 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b798c782-f5d4-4b96-a77a-3526ddb2b90d None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:08.285964 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59891aa4-efbc-4223-a1b9-827c60bb35a3 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:08.370705 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602db26b-0172-475b-a0cb-f4c6d7d338a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:08.371583 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602db26b-0172-475b-a0cb-f4c6d7d338a7 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:08.371866 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602db26b-0172-475b-a0cb-f4c6d7d338a7 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:08.372153 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602db26b-0172-475b-a0cb-f4c6d7d338a7 None admin] PATH_INFO: `/v3/projects/92337f7cabcc48ff8ca007298d1c3d78` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:08.376437 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602db26b-0172-475b-a0cb-f4c6d7d338a7 None admin] RBAC: Authorizing `identity:get_project(project_id=92337f7cabcc48ff8ca007298d1c3d78)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:08.377922 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602db26b-0172-475b-a0cb-f4c6d7d338a7 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:08.379952 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 74/149] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:44:08 2022] GET /identity/v3/projects/92337f7cabcc48ff8ca007298d1c3d78 => generated 298 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:08.396746 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602db26b-0172-475b-a0cb-f4c6d7d338a7 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:08.515254 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602db26b-0172-475b-a0cb-f4c6d7d338a7 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:08.526036 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b798c782-f5d4-4b96-a77a-3526ddb2b90d None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:08.560901 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd253806-7366-4476-98ff-8f9d75be632d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:08.565328 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd253806-7366-4476-98ff-8f9d75be632d None admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:08.565816 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd253806-7366-4476-98ff-8f9d75be632d None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:08.566294 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd253806-7366-4476-98ff-8f9d75be632d None admin] PATH_INFO: `/v3/projects/92337f7cabcc48ff8ca007298d1c3d78/users/cdcebaa21c554387a0c7293ad5033b28/roles/e5c0610b69d640e595dccba09f44b81e` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:08.572691 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd253806-7366-4476-98ff-8f9d75be632d None admin] RBAC: Authorizing `identity:create_grant(project_id=92337f7cabcc48ff8ca007298d1c3d78, user_id=cdcebaa21c554387a0c7293ad5033b28, role_id=e5c0610b69d640e595dccba09f44b81e)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:08.575340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd253806-7366-4476-98ff-8f9d75be632d None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:08.612376 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 75/150] 10.222.0.23 () {58 vars in 1439 bytes} [Fri May 6 14:44:08 2022] PUT /identity/v3/projects/92337f7cabcc48ff8ca007298d1c3d78/users/cdcebaa21c554387a0c7293ad5033b28/roles/e5c0610b69d640e595dccba09f44b81e => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:44:08.623946 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be332c45-67de-44ea-9824-9135a5bedb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:08.625277 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be332c45-67de-44ea-9824-9135a5bedb43 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:08.625772 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be332c45-67de-44ea-9824-9135a5bedb43 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:08.626253 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be332c45-67de-44ea-9824-9135a5bedb43 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:08.647936 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be332c45-67de-44ea-9824-9135a5bedb43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:08.664282 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-be332c45-67de-44ea-9824-9135a5bedb43 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 06 14:44:08.664282 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:08.664282 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:08.664282 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:08.664282 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:08.664282 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:08.664282 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:08.664282 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:08.664282 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:08.664282 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:08.664282 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:08.664282 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:08.664282 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:08.664282 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:08.664282 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:08.664282 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:08.669965 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:08.669965 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:08.669965 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:08.669965 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:08.669965 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:08.669965 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:08.669965 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:08.669965 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:08.669965 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:08.669965 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:08.669965 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:08.669965 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:44:08.669965 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:08.669965 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:08.669965 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:08.669965 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:08.673008 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:08.673008 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:08.673008 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:08.673008 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:08.673008 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:08.673008 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:08.673008 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:08.673008 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:08.673008 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:08.673008 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:08.673008 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 06 14:44:08.673008 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:44:08.673008 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 76/151] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:44:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 411 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:44:08.683514 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd253806-7366-4476-98ff-8f9d75be632d None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:08.879073 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd253806-7366-4476-98ff-8f9d75be632d None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:08.941468 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d5e835-f6d6-438f-84a4-b505272020d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:08.942602 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5e835-f6d6-438f-84a4-b505272020d1 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:08.943096 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5e835-f6d6-438f-84a4-b505272020d1 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:08.943581 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5e835-f6d6-438f-84a4-b505272020d1 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:08.944982 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d5e835-f6d6-438f-84a4-b505272020d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:08.945531 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d5e835-f6d6-438f-84a4-b505272020d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:08.948928 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d5e835-f6d6-438f-84a4-b505272020d1 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:08.970557 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 76/152] 10.222.0.23 () {56 vars in 1101 bytes} [Fri May 6 14:44:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:08.979868 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be332c45-67de-44ea-9824-9135a5bedb43 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:09.173700 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be332c45-67de-44ea-9824-9135a5bedb43 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:09.240876 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba08fb21-e72a-41d9-9cf2-10dec849b1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:09.242352 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba08fb21-e72a-41d9-9cf2-10dec849b1da None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:09.243493 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba08fb21-e72a-41d9-9cf2-10dec849b1da None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:09.244267 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba08fb21-e72a-41d9-9cf2-10dec849b1da None admin] PATH_INFO: `/v3/users/f26c0bc8ce6d4932bba691c04bec3ded` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:09.250262 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba08fb21-e72a-41d9-9cf2-10dec849b1da None admin] RBAC: Authorizing `identity:get_user(user_id=f26c0bc8ce6d4932bba691c04bec3ded)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:09.252971 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba08fb21-e72a-41d9-9cf2-10dec849b1da None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:09.256144 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 77/153] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:44:08 2022] GET /identity/v3/users/f26c0bc8ce6d4932bba691c04bec3ded => generated 275 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:09.266179 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d5e835-f6d6-438f-84a4-b505272020d1 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:09.459078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d5e835-f6d6-438f-84a4-b505272020d1 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:09.529529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227822ab-967f-462b-a6fc-80a25808903e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:09.530709 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227822ab-967f-462b-a6fc-80a25808903e None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:09.531397 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227822ab-967f-462b-a6fc-80a25808903e None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:09.531902 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227822ab-967f-462b-a6fc-80a25808903e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:09.674311 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227822ab-967f-462b-a6fc-80a25808903e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:09.677113 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227822ab-967f-462b-a6fc-80a25808903e None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:09.791003 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 77/154] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:44:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 06 14:44:09.796161 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba08fb21-e72a-41d9-9cf2-10dec849b1da None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:09.908959 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba08fb21-e72a-41d9-9cf2-10dec849b1da None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:09.957604 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c290ed-829b-4667-912b-e07088f7f42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:09.959494 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c290ed-829b-4667-912b-e07088f7f42c None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:09.960296 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c290ed-829b-4667-912b-e07088f7f42c None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:09.961064 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c290ed-829b-4667-912b-e07088f7f42c None admin] PATH_INFO: `/v3/projects/fd4139f9cd2643d3b04dfeda92c67546` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:09.964928 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c290ed-829b-4667-912b-e07088f7f42c None admin] RBAC: Authorizing `identity:get_project(project_id=fd4139f9cd2643d3b04dfeda92c67546)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:09.967767 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c290ed-829b-4667-912b-e07088f7f42c None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:09.971223 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 78/155] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:44:09 2022] GET /identity/v3/projects/fd4139f9cd2643d3b04dfeda92c67546 => generated 312 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:09.977430 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227822ab-967f-462b-a6fc-80a25808903e None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:10.003036 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258aeded-fb7d-40fe-b7ee-0743711a62eb None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:10.003931 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258aeded-fb7d-40fe-b7ee-0743711a62eb None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:10.004721 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258aeded-fb7d-40fe-b7ee-0743711a62eb None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:10.012773 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 79/156] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:10.019104 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa1f55-9ea4-4e6a-ab48-364e13f2c8e4 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:10.019898 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa1f55-9ea4-4e6a-ab48-364e13f2c8e4 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:10.020678 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa1f55-9ea4-4e6a-ab48-364e13f2c8e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:10.066385 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-82fa1f55-9ea4-4e6a-ab48-364e13f2c8e4 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:10.068224 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-82fa1f55-9ea4-4e6a-ab48-364e13f2c8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:10.259695 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227822ab-967f-462b-a6fc-80a25808903e None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:10.263227 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 80/157] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 06 14:44:10.266753 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf2a917-c9f1-470e-9322-7e51a2c358e8 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:10.267232 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf2a917-c9f1-470e-9322-7e51a2c358e8 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:10.267709 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf2a917-c9f1-470e-9322-7e51a2c358e8 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:10.270636 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 81/158] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:10.281193 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b91809-19d3-4d0b-8f15-f07d73e1854f None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:10.281663 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b91809-19d3-4d0b-8f15-f07d73e1854f None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:10.282146 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b91809-19d3-4d0b-8f15-f07d73e1854f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:10.327524 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-05b91809-19d3-4d0b-8f15-f07d73e1854f None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:10.329583 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-05b91809-19d3-4d0b-8f15-f07d73e1854f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:10.337311 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d354de5a-3122-4f79-9acd-96a78de2e703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:10.337833 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d354de5a-3122-4f79-9acd-96a78de2e703 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:10.338118 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d354de5a-3122-4f79-9acd-96a78de2e703 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:10.338394 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d354de5a-3122-4f79-9acd-96a78de2e703 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:10.339423 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d354de5a-3122-4f79-9acd-96a78de2e703 None admin] RBAC: Adding query filter params (role.id=5f76e2e820b64d14b893c679427f4bed, scope.project.id=fd4139f9cd2643d3b04dfeda92c67546, user.id=f26c0bc8ce6d4932bba691c04bec3ded) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:10.339741 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d354de5a-3122-4f79-9acd-96a78de2e703 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:10.341248 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d354de5a-3122-4f79-9acd-96a78de2e703 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:10.358242 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 78/159] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:44:09 2022] GET /identity/v3/role_assignments?user.id=f26c0bc8ce6d4932bba691c04bec3ded&scope.project.id=fd4139f9cd2643d3b04dfeda92c67546&role.id=5f76e2e820b64d14b893c679427f4bed => generated 608 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:10.358763 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d354de5a-3122-4f79-9acd-96a78de2e703 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:10.478859 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 82/160] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 06 14:44:10.511339 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d354de5a-3122-4f79-9acd-96a78de2e703 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:10.534869 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b91809-19d3-4d0b-8f15-f07d73e1854f None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:10.570563 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71f7610-ec49-43cb-9014-0a481ea29639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:10.571590 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f7610-ec49-43cb-9014-0a481ea29639 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:10.571829 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f7610-ec49-43cb-9014-0a481ea29639 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:10.572068 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f7610-ec49-43cb-9014-0a481ea29639 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:10.596430 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71f7610-ec49-43cb-9014-0a481ea29639 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:10.608028 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-a71f7610-ec49-43cb-9014-0a481ea29639 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 06 14:44:10.608028 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:10.608028 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:10.608028 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:10.608028 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:10.608028 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:10.608028 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:10.608028 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:10.608028 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:10.608028 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:10.608028 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:10.608028 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:10.608028 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:10.608028 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:10.608028 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:10.608028 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:10.611446 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:10.611446 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:10.611446 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:10.611446 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:10.611446 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:10.611446 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:10.611446 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:10.611446 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:10.611446 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:10.611446 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:10.611446 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:10.611446 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:44:10.611446 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:10.611446 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:10.611446 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:10.611446 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:10.614446 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:10.614446 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:10.614446 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:10.614446 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:10.614446 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:10.614446 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:10.614446 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:10.614446 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:10.614446 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:10.614446 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:10.614446 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 06 14:44:10.614446 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:44:10.614446 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 79/161] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:44:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:10.623004 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71f7610-ec49-43cb-9014-0a481ea29639 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:10.714291 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b91809-19d3-4d0b-8f15-f07d73e1854f None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:10.779630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb8fd2-5039-4c0e-aef4-5b8a3118beca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:10.780506 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb8fd2-5039-4c0e-aef4-5b8a3118beca None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:10.781072 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb8fd2-5039-4c0e-aef4-5b8a3118beca None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:10.781362 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb8fd2-5039-4c0e-aef4-5b8a3118beca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:10.800140 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bb8fd2-5039-4c0e-aef4-5b8a3118beca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:10.811754 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71f7610-ec49-43cb-9014-0a481ea29639 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:10.814383 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-a4bb8fd2-5039-4c0e-aef4-5b8a3118beca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 06 14:44:10.814383 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:10.814383 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:10.814383 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:10.814383 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:10.814383 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:10.814383 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:10.814383 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:10.814383 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:10.814383 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:10.814383 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:10.814383 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:10.814383 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:10.814383 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:10.814383 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:10.814383 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:10.817698 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:10.817698 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:10.817698 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:10.817698 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:10.817698 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:10.817698 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:10.817698 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:10.817698 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:10.817698 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:10.817698 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:10.817698 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:10.817698 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:44:10.817698 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:10.817698 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:10.817698 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:10.817698 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:10.822344 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:10.822344 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:10.822344 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:10.822344 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:10.822344 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:10.822344 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:10.822344 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:10.822344 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:10.822344 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:10.822344 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:10.822344 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 06 14:44:10.822344 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:44:10.822344 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 83/162] 10.222.0.23 () {56 vars in 1114 bytes} [Fri May 6 14:44:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:44:10.828775 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb8fd2-5039-4c0e-aef4-5b8a3118beca None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:10.871148 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb29a677-2f99-473f-9957-78ca6806e424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:10.871733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb29a677-2f99-473f-9957-78ca6806e424 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:10.872020 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb29a677-2f99-473f-9957-78ca6806e424 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:10.872299 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb29a677-2f99-473f-9957-78ca6806e424 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:10.873878 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb29a677-2f99-473f-9957-78ca6806e424 None admin] RBAC: Adding query filter params (name=member) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:10.874241 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb29a677-2f99-473f-9957-78ca6806e424 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:10.875644 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb29a677-2f99-473f-9957-78ca6806e424 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:10.892953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 80/163] 10.222.0.23 () {56 vars in 1101 bytes} [Fri May 6 14:44:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:10.899815 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb29a677-2f99-473f-9957-78ca6806e424 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:11.012162 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb8fd2-5039-4c0e-aef4-5b8a3118beca None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:11.073119 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77bf4e9-cc62-42bc-97f2-f5d7bcbbe1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:11.073796 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77bf4e9-cc62-42bc-97f2-f5d7bcbbe1ca None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:11.074078 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77bf4e9-cc62-42bc-97f2-f5d7bcbbe1ca None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:11.074358 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77bf4e9-cc62-42bc-97f2-f5d7bcbbe1ca None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:11.075116 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77bf4e9-cc62-42bc-97f2-f5d7bcbbe1ca None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:11.075428 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77bf4e9-cc62-42bc-97f2-f5d7bcbbe1ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:11.076824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77bf4e9-cc62-42bc-97f2-f5d7bcbbe1ca None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:11.085107 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb29a677-2f99-473f-9957-78ca6806e424 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:11.098272 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 84/164] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:44:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:11.103997 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77bf4e9-cc62-42bc-97f2-f5d7bcbbe1ca None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:11.146536 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7413ee-5ff2-407f-8569-49ebd4a217c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:11.147325 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7413ee-5ff2-407f-8569-49ebd4a217c3 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:11.147613 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7413ee-5ff2-407f-8569-49ebd4a217c3 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:11.147908 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7413ee-5ff2-407f-8569-49ebd4a217c3 None admin] PATH_INFO: `/v3/users/f26c0bc8ce6d4932bba691c04bec3ded` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:11.149894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7413ee-5ff2-407f-8569-49ebd4a217c3 None admin] RBAC: Authorizing `identity:get_user(user_id=f26c0bc8ce6d4932bba691c04bec3ded)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:11.164848 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7413ee-5ff2-407f-8569-49ebd4a217c3 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:11.164848 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 81/165] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:44:10 2022] GET /identity/v3/users/f26c0bc8ce6d4932bba691c04bec3ded => generated 275 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:11.164848 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7413ee-5ff2-407f-8569-49ebd4a217c3 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:11.357818 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77bf4e9-cc62-42bc-97f2-f5d7bcbbe1ca None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:11.418989 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7413ee-5ff2-407f-8569-49ebd4a217c3 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:11.435512 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0eaf02-ee7f-4b0a-96c6-e1db6585a884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:11.436401 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0eaf02-ee7f-4b0a-96c6-e1db6585a884 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:11.436708 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0eaf02-ee7f-4b0a-96c6-e1db6585a884 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:11.436985 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0eaf02-ee7f-4b0a-96c6-e1db6585a884 None admin] PATH_INFO: `/v3/users/f26c0bc8ce6d4932bba691c04bec3ded` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:11.439903 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0eaf02-ee7f-4b0a-96c6-e1db6585a884 None admin] RBAC: Authorizing `identity:get_user(user_id=f26c0bc8ce6d4932bba691c04bec3ded)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:11.442533 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0eaf02-ee7f-4b0a-96c6-e1db6585a884 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:11.447377 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 85/166] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:44:11 2022] GET /identity/v3/users/f26c0bc8ce6d4932bba691c04bec3ded => generated 275 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:11.453165 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0eaf02-ee7f-4b0a-96c6-e1db6585a884 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:11.494851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e1bde-2ce7-42fe-b6d3-fe52fb162c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:11.495614 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e1bde-2ce7-42fe-b6d3-fe52fb162c03 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:11.495902 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e1bde-2ce7-42fe-b6d3-fe52fb162c03 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:11.496177 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e1bde-2ce7-42fe-b6d3-fe52fb162c03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:11.635846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0eaf02-ee7f-4b0a-96c6-e1db6585a884 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:11.661765 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252e1bde-2ce7-42fe-b6d3-fe52fb162c03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:11.663274 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252e1bde-2ce7-42fe-b6d3-fe52fb162c03 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:11.693185 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34270893-e8e1-4a6f-9828-6fdc04bc25e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:11.693833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34270893-e8e1-4a6f-9828-6fdc04bc25e2 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:11.694122 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34270893-e8e1-4a6f-9828-6fdc04bc25e2 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:11.694390 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34270893-e8e1-4a6f-9828-6fdc04bc25e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:11.816969 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34270893-e8e1-4a6f-9828-6fdc04bc25e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:11.819549 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34270893-e8e1-4a6f-9828-6fdc04bc25e2 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:11.884728 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 82/167] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:44:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 722 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 06 14:44:11.889786 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e1bde-2ce7-42fe-b6d3-fe52fb162c03 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:11.988318 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 86/168] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:44:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 06 14:44:11.998209 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34270893-e8e1-4a6f-9828-6fdc04bc25e2 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:12.204381 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e1bde-2ce7-42fe-b6d3-fe52fb162c03 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:12.246756 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1c2eb5-d39b-4c95-a265-128f1b04845c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:12.247523 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c2eb5-d39b-4c95-a265-128f1b04845c None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:12.247813 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c2eb5-d39b-4c95-a265-128f1b04845c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:12.248089 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c2eb5-d39b-4c95-a265-128f1b04845c None admin] PATH_INFO: `/v3/projects/92337f7cabcc48ff8ca007298d1c3d78` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:12.253327 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1c2eb5-d39b-4c95-a265-128f1b04845c None admin] RBAC: Authorizing `identity:get_project(project_id=92337f7cabcc48ff8ca007298d1c3d78)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:12.253793 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34270893-e8e1-4a6f-9828-6fdc04bc25e2 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:12.254799 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1c2eb5-d39b-4c95-a265-128f1b04845c None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:12.256514 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 83/169] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:44:11 2022] GET /identity/v3/projects/92337f7cabcc48ff8ca007298d1c3d78 => generated 298 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:12.257799 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed1503-0bab-42ed-a105-e74387118df0 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:12.258288 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed1503-0bab-42ed-a105-e74387118df0 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:12.258288 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed1503-0bab-42ed-a105-e74387118df0 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:12.259697 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 84/170] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:12.262603 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed1503-0bab-42ed-a105-e74387118df0 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:12.321610 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6634d5-a551-499e-9004-00212ca307d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:12.322891 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6634d5-a551-499e-9004-00212ca307d4 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:12.323365 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6634d5-a551-499e-9004-00212ca307d4 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:12.323849 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6634d5-a551-499e-9004-00212ca307d4 None admin] PATH_INFO: `/v3/projects/92337f7cabcc48ff8ca007298d1c3d78` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:12.333272 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6634d5-a551-499e-9004-00212ca307d4 None admin] RBAC: Authorizing `identity:get_project(project_id=92337f7cabcc48ff8ca007298d1c3d78)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:12.342605 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6634d5-a551-499e-9004-00212ca307d4 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:12.344831 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 87/171] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:44:11 2022] GET /identity/v3/projects/92337f7cabcc48ff8ca007298d1c3d78 => generated 298 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:12.346061 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5571ba-a89d-4a13-9a51-fbeb03b0bf3c None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:12.346348 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5571ba-a89d-4a13-9a51-fbeb03b0bf3c None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:12.346628 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5571ba-a89d-4a13-9a51-fbeb03b0bf3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:12.386566 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-ca5571ba-a89d-4a13-9a51-fbeb03b0bf3c None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:12.388477 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ca5571ba-a89d-4a13-9a51-fbeb03b0bf3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:12.499683 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed1503-0bab-42ed-a105-e74387118df0 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:12.539319 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6bc4e9-8ac6-4a7d-9eab-9bbcf38512d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:12.540288 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6bc4e9-8ac6-4a7d-9eab-9bbcf38512d6 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:12.540754 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6bc4e9-8ac6-4a7d-9eab-9bbcf38512d6 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:12.541169 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6bc4e9-8ac6-4a7d-9eab-9bbcf38512d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:12.542157 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6bc4e9-8ac6-4a7d-9eab-9bbcf38512d6 None admin] RBAC: Adding query filter params (role.id=5f76e2e820b64d14b893c679427f4bed, scope.project.id=92337f7cabcc48ff8ca007298d1c3d78, user.id=f26c0bc8ce6d4932bba691c04bec3ded) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:12.542615 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6bc4e9-8ac6-4a7d-9eab-9bbcf38512d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:12.544252 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6bc4e9-8ac6-4a7d-9eab-9bbcf38512d6 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:12.551538 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 85/172] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:44:12 2022] GET /identity/v3/role_assignments?user.id=f26c0bc8ce6d4932bba691c04bec3ded&scope.project.id=92337f7cabcc48ff8ca007298d1c3d78&role.id=5f76e2e820b64d14b893c679427f4bed => generated 608 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:12.552549 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6bc4e9-8ac6-4a7d-9eab-9bbcf38512d6 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:12.595035 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 88/173] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:44:12.699933 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6bc4e9-8ac6-4a7d-9eab-9bbcf38512d6 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:12.702642 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5571ba-a89d-4a13-9a51-fbeb03b0bf3c None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:12.735230 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a39e20c-82ea-472e-9032-a9ccf073f570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:12.735906 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39e20c-82ea-472e-9032-a9ccf073f570 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:12.736238 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39e20c-82ea-472e-9032-a9ccf073f570 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:12.736513 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39e20c-82ea-472e-9032-a9ccf073f570 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:12.737559 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a39e20c-82ea-472e-9032-a9ccf073f570 None admin] RBAC: Adding query filter params (role.id=d328f8040f2247d083309d839d2ff947, scope.project.id=92337f7cabcc48ff8ca007298d1c3d78, user.id=f26c0bc8ce6d4932bba691c04bec3ded) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:12.738021 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a39e20c-82ea-472e-9032-a9ccf073f570 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:12.739675 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a39e20c-82ea-472e-9032-a9ccf073f570 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:12.746774 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 86/174] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:44:12 2022] GET /identity/v3/role_assignments?user.id=f26c0bc8ce6d4932bba691c04bec3ded&scope.project.id=92337f7cabcc48ff8ca007298d1c3d78&role.id=d328f8040f2247d083309d839d2ff947 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:12.748339 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b60cf-edaf-4b1c-8835-ccc47edbbba5 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:12.748783 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b60cf-edaf-4b1c-8835-ccc47edbbba5 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:12.749198 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b60cf-edaf-4b1c-8835-ccc47edbbba5 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:12.750627 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 87/175] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:12.758353 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110d619-293c-4ebd-9da0-f3e164f8b099 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:12.759084 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110d619-293c-4ebd-9da0-f3e164f8b099 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:12.759511 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110d619-293c-4ebd-9da0-f3e164f8b099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:12.790647 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-3110d619-293c-4ebd-9da0-f3e164f8b099 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:12.792106 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-3110d619-293c-4ebd-9da0-f3e164f8b099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:12.943399 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5571ba-a89d-4a13-9a51-fbeb03b0bf3c None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:12.962843 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 88/176] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 06 14:44:13.004754 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3110d619-293c-4ebd-9da0-f3e164f8b099 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:13.011688 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6639fdfe-cb41-4762-bc0e-bac871d3969e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:13.012503 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6639fdfe-cb41-4762-bc0e-bac871d3969e None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:13.012921 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6639fdfe-cb41-4762-bc0e-bac871d3969e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:13.013199 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6639fdfe-cb41-4762-bc0e-bac871d3969e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:13.032491 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6639fdfe-cb41-4762-bc0e-bac871d3969e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:13.041646 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-6639fdfe-cb41-4762-bc0e-bac871d3969e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:44:13.041646 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:13.041646 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:13.041646 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:13.041646 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:13.041646 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:13.041646 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:13.041646 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:13.041646 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:13.041646 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:13.041646 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:13.041646 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:13.041646 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:13.041646 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:13.041646 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:13.041646 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:13.042695 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:13.042695 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:13.042695 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:13.042695 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:13.042695 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:13.042695 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:13.042695 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:13.042695 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:13.042695 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:13.042695 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:13.042695 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:13.042695 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:44:13.042695 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:13.042695 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:13.042695 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:13.042695 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:13.043487 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:13.043487 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:13.043487 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:13.043487 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:13.043487 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:13.043487 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:13.043487 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:13.043487 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:13.043487 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:13.043487 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:13.043487 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:44:13.043487 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:44:13.043487 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 89/177] 10.222.0.23 () {56 vars in 1096 bytes} [Fri May 6 14:44:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:44:13.051621 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6639fdfe-cb41-4762-bc0e-bac871d3969e None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:13.150870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3110d619-293c-4ebd-9da0-f3e164f8b099 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:13.233251 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788aea38-f870-423e-b0d1-c23fb7cdd45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:13.234068 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788aea38-f870-423e-b0d1-c23fb7cdd45c None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:13.234347 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788aea38-f870-423e-b0d1-c23fb7cdd45c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:13.234621 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788aea38-f870-423e-b0d1-c23fb7cdd45c None admin] PATH_INFO: `/v3/domains/default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:13.236913 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788aea38-f870-423e-b0d1-c23fb7cdd45c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:13.238388 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788aea38-f870-423e-b0d1-c23fb7cdd45c None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:13.240020 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 89/178] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:44:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:13.244574 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6639fdfe-cb41-4762-bc0e-bac871d3969e None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:13.246960 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788aea38-f870-423e-b0d1-c23fb7cdd45c None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:13.312187 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4f35a1-bcb2-4b99-a88e-3216e1c348a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:13.312822 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4f35a1-bcb2-4b99-a88e-3216e1c348a6 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:13.313107 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4f35a1-bcb2-4b99-a88e-3216e1c348a6 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:13.313385 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4f35a1-bcb2-4b99-a88e-3216e1c348a6 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:13.314145 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4f35a1-bcb2-4b99-a88e-3216e1c348a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:13.314453 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4f35a1-bcb2-4b99-a88e-3216e1c348a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:13.315874 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4f35a1-bcb2-4b99-a88e-3216e1c348a6 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:13.330140 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 90/179] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:44:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:13.335939 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4f35a1-bcb2-4b99-a88e-3216e1c348a6 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:13.426363 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788aea38-f870-423e-b0d1-c23fb7cdd45c None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:13.469762 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ca8c0-a4ed-46cb-ac71-1e21050955e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:13.470772 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ca8c0-a4ed-46cb-ac71-1e21050955e2 None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:13.471202 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ca8c0-a4ed-46cb-ac71-1e21050955e2 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:13.471619 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ca8c0-a4ed-46cb-ac71-1e21050955e2 None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:13.472469 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385ca8c0-a4ed-46cb-ac71-1e21050955e2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:13.474083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385ca8c0-a4ed-46cb-ac71-1e21050955e2 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:13.485411 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4f35a1-bcb2-4b99-a88e-3216e1c348a6 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:13.506545 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 90/180] 10.222.0.23 () {60 vars in 1132 bytes} [Fri May 6 14:44:13 2022] POST /identity/v3/users => generated 297 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:44:13.509469 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb576e8a-cb25-44f7-894c-ac30f81640f1 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:13.510010 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb576e8a-cb25-44f7-894c-ac30f81640f1 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:13.510460 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb576e8a-cb25-44f7-894c-ac30f81640f1 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:13.512136 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 91/181] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:13.522167 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483ab17-45c9-49d4-b396-8f941636b67a None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:13.522745 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483ab17-45c9-49d4-b396-8f941636b67a None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:13.523204 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483ab17-45c9-49d4-b396-8f941636b67a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:13.534715 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c416a9f-ad5e-4067-b236-7528ae98092b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:13.535267 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c416a9f-ad5e-4067-b236-7528ae98092b None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:13.535623 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c416a9f-ad5e-4067-b236-7528ae98092b None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:13.540627 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c416a9f-ad5e-4067-b236-7528ae98092b None admin] PATH_INFO: `/v3/users/66f1687219494ee0a402084110e075fd` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:13.540627 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c416a9f-ad5e-4067-b236-7528ae98092b None admin] RBAC: Authorizing `identity:get_user(user_id=66f1687219494ee0a402084110e075fd)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:13.540627 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c416a9f-ad5e-4067-b236-7528ae98092b None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:13.541676 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 91/182] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:44:13 2022] GET /identity/v3/users/66f1687219494ee0a402084110e075fd => generated 247 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:13.547560 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c416a9f-ad5e-4067-b236-7528ae98092b None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:13.553833 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-c483ab17-45c9-49d4-b396-8f941636b67a None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:13.563086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-c483ab17-45c9-49d4-b396-8f941636b67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:13.661814 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c416a9f-ad5e-4067-b236-7528ae98092b None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:13.667078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 92/183] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:44:13.668688 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e458fdd8-572c-4f68-a315-1f66d5b556fa None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:13.669120 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e458fdd8-572c-4f68-a315-1f66d5b556fa None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:13.669537 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e458fdd8-572c-4f68-a315-1f66d5b556fa None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:13.670953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 93/184] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:13.676329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52cebc-3867-4185-8e4f-06205bee7b95 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:13.676822 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52cebc-3867-4185-8e4f-06205bee7b95 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:13.677244 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52cebc-3867-4185-8e4f-06205bee7b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:13.694897 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-ba52cebc-3867-4185-8e4f-06205bee7b95 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:13.697139 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-ba52cebc-3867-4185-8e4f-06205bee7b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:13.701412 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758c781-6ef1-4ac4-9936-5445735286a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:13.701763 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758c781-6ef1-4ac4-9936-5445735286a4 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:13.702038 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758c781-6ef1-4ac4-9936-5445735286a4 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:13.702373 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758c781-6ef1-4ac4-9936-5445735286a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:13.770454 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 94/185] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 06 14:44:13.776622 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba52cebc-3867-4185-8e4f-06205bee7b95 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:13.780306 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6758c781-6ef1-4ac4-9936-5445735286a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:13.781470 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6758c781-6ef1-4ac4-9936-5445735286a4 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:13.890063 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba52cebc-3867-4185-8e4f-06205bee7b95 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:13.902914 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 92/186] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:44:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 06 14:44:13.903774 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758c781-6ef1-4ac4-9936-5445735286a4 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:13.928375 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeb0b22-7294-424e-b4b2-54f4cc35503f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:13.929299 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb0b22-7294-424e-b4b2-54f4cc35503f None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:13.929584 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb0b22-7294-424e-b4b2-54f4cc35503f None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:13.929918 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb0b22-7294-424e-b4b2-54f4cc35503f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:13.946150 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeb0b22-7294-424e-b4b2-54f4cc35503f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:13.954413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-deeb0b22-7294-424e-b4b2-54f4cc35503f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 06 14:44:13.954413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:13.954413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:13.954413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:13.954413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:13.954413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:13.954413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:13.954413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:13.954413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:13.954413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:13.954413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:13.954413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:13.954413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:13.954413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:13.954413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:13.954413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:13.955322 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:13.955322 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:13.955322 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:13.955322 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:13.955322 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:13.955322 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:13.955322 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:13.955322 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:13.955322 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:13.955322 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:13.955322 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:13.955322 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:44:13.955322 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:13.955322 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:13.955322 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:13.955322 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:13.956110 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:13.956110 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:13.956110 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:13.956110 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:13.956110 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:13.956110 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:13.956110 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:13.956110 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:13.956110 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:13.956110 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:13.956110 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 06 14:44:13.956110 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:44:13.956110 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 95/187] 10.222.0.23 () {56 vars in 1114 bytes} [Fri May 6 14:44:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:13.956784 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeb0b22-7294-424e-b4b2-54f4cc35503f None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:14.037163 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758c781-6ef1-4ac4-9936-5445735286a4 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:14.072625 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353a413-e8bb-4104-be4e-3f6a3c65e492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:14.073445 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353a413-e8bb-4104-be4e-3f6a3c65e492 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:14.073717 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353a413-e8bb-4104-be4e-3f6a3c65e492 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:14.073994 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353a413-e8bb-4104-be4e-3f6a3c65e492 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:14.078599 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeb0b22-7294-424e-b4b2-54f4cc35503f None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:14.085198 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f353a413-e8bb-4104-be4e-3f6a3c65e492 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:14.094463 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-f353a413-e8bb-4104-be4e-3f6a3c65e492 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:44:14.094463 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:14.094463 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:14.094463 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:14.094463 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:14.094463 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:14.094463 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:14.094463 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:14.094463 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:14.094463 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:14.094463 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:14.094463 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:14.094463 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:14.094463 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:14.094463 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:14.094463 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:14.095371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:14.095371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:14.095371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:14.095371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:14.095371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:14.095371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:14.095371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:14.095371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:14.095371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:14.095371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:14.095371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:14.095371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:44:14.095371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:14.095371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:14.095371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:14.095371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:14.096138 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:14.096138 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:14.096138 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:14.096138 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:14.096138 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:14.096138 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:14.096138 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:14.096138 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:14.096138 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:14.096138 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:14.096138 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:44:14.096138 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:44:14.096138 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 93/188] 10.222.0.23 () {56 vars in 1096 bytes} [Fri May 6 14:44:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:14.096810 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353a413-e8bb-4104-be4e-3f6a3c65e492 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:14.118991 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ba66a-6bc4-4d41-b950-896b919800a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:14.120266 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ba66a-6bc4-4d41-b950-896b919800a1 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:14.120775 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ba66a-6bc4-4d41-b950-896b919800a1 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:14.121277 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ba66a-6bc4-4d41-b950-896b919800a1 None admin] PATH_INFO: `/v3/projects/92337f7cabcc48ff8ca007298d1c3d78` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:14.122736 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ba66a-6bc4-4d41-b950-896b919800a1 None admin] RBAC: Authorizing `identity:get_project(project_id=92337f7cabcc48ff8ca007298d1c3d78)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:14.124436 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ba66a-6bc4-4d41-b950-896b919800a1 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:14.126414 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 96/189] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:44:13 2022] GET /identity/v3/projects/92337f7cabcc48ff8ca007298d1c3d78 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:14.127463 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ba66a-6bc4-4d41-b950-896b919800a1 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:14.212687 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353a413-e8bb-4104-be4e-3f6a3c65e492 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:14.238152 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ba66a-6bc4-4d41-b950-896b919800a1 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:14.247873 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d2cce-6b75-4403-93ea-16a8998fcbef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:14.248562 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d2cce-6b75-4403-93ea-16a8998fcbef None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:14.249141 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d2cce-6b75-4403-93ea-16a8998fcbef None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:14.249141 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d2cce-6b75-4403-93ea-16a8998fcbef None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:14.249872 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7d2cce-6b75-4403-93ea-16a8998fcbef None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:14.250180 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7d2cce-6b75-4403-93ea-16a8998fcbef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:14.251566 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7d2cce-6b75-4403-93ea-16a8998fcbef None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:14.259859 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 94/190] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:44:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:14.264627 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d2cce-6b75-4403-93ea-16a8998fcbef None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:14.281794 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d1875-4897-47ab-a1d8-bc9519467378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:14.282836 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d1875-4897-47ab-a1d8-bc9519467378 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:14.283309 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d1875-4897-47ab-a1d8-bc9519467378 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:14.283771 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d1875-4897-47ab-a1d8-bc9519467378 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:14.284760 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d1875-4897-47ab-a1d8-bc9519467378 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:14.285254 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d1875-4897-47ab-a1d8-bc9519467378 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:14.286878 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d1875-4897-47ab-a1d8-bc9519467378 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:14.296548 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 97/191] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:44:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:14.297716 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d1875-4897-47ab-a1d8-bc9519467378 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:14.367077 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d2cce-6b75-4403-93ea-16a8998fcbef None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:14.406375 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e62d86-3315-43e9-a287-3f5eca362348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:14.407040 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e62d86-3315-43e9-a287-3f5eca362348 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:14.407326 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e62d86-3315-43e9-a287-3f5eca362348 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:14.407605 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e62d86-3315-43e9-a287-3f5eca362348 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:14.408498 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e62d86-3315-43e9-a287-3f5eca362348 None admin] RBAC: Adding query filter params (role.id=6dcd35d1a83748759d6ef9f026077835, scope.project.id=92337f7cabcc48ff8ca007298d1c3d78, user.id=66f1687219494ee0a402084110e075fd) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:14.408789 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e62d86-3315-43e9-a287-3f5eca362348 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:14.409050 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d1875-4897-47ab-a1d8-bc9519467378 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:14.410250 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e62d86-3315-43e9-a287-3f5eca362348 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:14.418227 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 95/192] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:44:14 2022] GET /identity/v3/role_assignments?user.id=66f1687219494ee0a402084110e075fd&scope.project.id=92337f7cabcc48ff8ca007298d1c3d78&role.id=6dcd35d1a83748759d6ef9f026077835 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:14.418973 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e62d86-3315-43e9-a287-3f5eca362348 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:14.448931 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca7ecac-af0b-4b47-a7f0-9925692aa887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:14.450314 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7ecac-af0b-4b47-a7f0-9925692aa887 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:14.450823 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7ecac-af0b-4b47-a7f0-9925692aa887 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:14.451315 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7ecac-af0b-4b47-a7f0-9925692aa887 None admin] PATH_INFO: `/v3/users/b7c1d929e29d48d4bd5322d408301fd8` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:14.477653 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca7ecac-af0b-4b47-a7f0-9925692aa887 None admin] RBAC: Authorizing `identity:get_user(user_id=b7c1d929e29d48d4bd5322d408301fd8)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:14.479660 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca7ecac-af0b-4b47-a7f0-9925692aa887 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:14.486912 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 98/193] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:44:14 2022] GET /identity/v3/users/b7c1d929e29d48d4bd5322d408301fd8 => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:14.488579 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813d91e-88a4-4ce6-9291-d3c6ed78ea88 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:14.489080 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813d91e-88a4-4ce6-9291-d3c6ed78ea88 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:14.489572 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813d91e-88a4-4ce6-9291-d3c6ed78ea88 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:14.491355 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 99/194] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:14.492387 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813d91e-88a4-4ce6-9291-d3c6ed78ea88 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:14.547699 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e62d86-3315-43e9-a287-3f5eca362348 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:14.587037 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e24436-3ecb-411c-82b9-b8df7b796e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:14.587975 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e24436-3ecb-411c-82b9-b8df7b796e27 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:14.588255 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e24436-3ecb-411c-82b9-b8df7b796e27 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:14.588530 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e24436-3ecb-411c-82b9-b8df7b796e27 None admin] PATH_INFO: `/v3/users/b7c1d929e29d48d4bd5322d408301fd8` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:14.590279 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e24436-3ecb-411c-82b9-b8df7b796e27 None admin] RBAC: Authorizing `identity:get_user(user_id=b7c1d929e29d48d4bd5322d408301fd8)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:14.591917 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e24436-3ecb-411c-82b9-b8df7b796e27 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:14.593993 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 96/195] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:44:14 2022] GET /identity/v3/users/b7c1d929e29d48d4bd5322d408301fd8 => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:14.595794 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934e493-e8a9-4150-a369-15b264bd984a None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:14.596084 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934e493-e8a9-4150-a369-15b264bd984a None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:14.596363 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934e493-e8a9-4150-a369-15b264bd984a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:14.615730 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-3934e493-e8a9-4150-a369-15b264bd984a None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:14.616967 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-3934e493-e8a9-4150-a369-15b264bd984a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:14.696694 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 97/196] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:44:14.697705 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934e493-e8a9-4150-a369-15b264bd984a None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:14.747867 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813d91e-88a4-4ce6-9291-d3c6ed78ea88 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:14.796992 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a929f156-01ef-4125-a5b6-16355ba85733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:14.797633 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929f156-01ef-4125-a5b6-16355ba85733 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:14.797914 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929f156-01ef-4125-a5b6-16355ba85733 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:14.798198 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929f156-01ef-4125-a5b6-16355ba85733 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:14.817652 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934e493-e8a9-4150-a369-15b264bd984a None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:14.861490 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1a0ec5-d4aa-471d-8113-3662ca839278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:14.862503 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a0ec5-d4aa-471d-8113-3662ca839278 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:14.862789 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a0ec5-d4aa-471d-8113-3662ca839278 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:14.863356 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a0ec5-d4aa-471d-8113-3662ca839278 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:14.873034 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a929f156-01ef-4125-a5b6-16355ba85733 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:14.874960 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a929f156-01ef-4125-a5b6-16355ba85733 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:14.937844 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1a0ec5-d4aa-471d-8113-3662ca839278 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:14.939342 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1a0ec5-d4aa-471d-8113-3662ca839278 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:14.978672 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 100/197] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:44:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 06 14:44:14.979456 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a929f156-01ef-4125-a5b6-16355ba85733 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:15.041839 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 98/198] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:44:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 06 14:44:15.042688 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1a0ec5-d4aa-471d-8113-3662ca839278 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:15.077889 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a929f156-01ef-4125-a5b6-16355ba85733 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:15.113031 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26631cd2-1ee5-45ef-b40e-db92fe67a671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:15.114604 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26631cd2-1ee5-45ef-b40e-db92fe67a671 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:15.114889 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26631cd2-1ee5-45ef-b40e-db92fe67a671 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:15.115166 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26631cd2-1ee5-45ef-b40e-db92fe67a671 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:15.128746 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26631cd2-1ee5-45ef-b40e-db92fe67a671 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:15.139497 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-26631cd2-1ee5-45ef-b40e-db92fe67a671 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 06 14:44:15.139497 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:15.139497 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:15.139497 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:15.139497 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:15.139497 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:15.139497 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:15.139497 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:15.139497 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:15.139497 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:15.139497 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:15.139497 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:15.139497 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:15.139497 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:15.139497 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:15.139497 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:15.141868 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:15.141868 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:15.141868 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:15.141868 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:15.141868 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:15.141868 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:15.141868 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:15.141868 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:15.141868 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:15.141868 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:15.141868 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:15.141868 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:44:15.141868 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:15.141868 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:15.141868 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:15.141868 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:15.143001 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:15.143001 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:15.143001 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:15.143001 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:15.143001 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:15.143001 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:15.143001 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:15.143001 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:15.143001 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:15.143001 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:15.143001 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 06 14:44:15.143001 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:44:15.143001 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 101/199] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:44:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:15.143001 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26631cd2-1ee5-45ef-b40e-db92fe67a671 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:15.148357 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1a0ec5-d4aa-471d-8113-3662ca839278 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:15.193480 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e06fab-636d-4874-8dab-1d400849c758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:15.194704 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e06fab-636d-4874-8dab-1d400849c758 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:15.195139 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e06fab-636d-4874-8dab-1d400849c758 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:15.195612 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e06fab-636d-4874-8dab-1d400849c758 None admin] PATH_INFO: `/v3/projects/643972018eac4175bcd3d9778028e93a` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:15.197056 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e06fab-636d-4874-8dab-1d400849c758 None admin] RBAC: Authorizing `identity:get_project(project_id=643972018eac4175bcd3d9778028e93a)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:15.199515 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e06fab-636d-4874-8dab-1d400849c758 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:15.201884 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 99/200] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:44:15 2022] GET /identity/v3/projects/643972018eac4175bcd3d9778028e93a => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:15.202751 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e06fab-636d-4874-8dab-1d400849c758 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:15.252794 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26631cd2-1ee5-45ef-b40e-db92fe67a671 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:15.288689 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d75d8df-0c65-4ca2-a4d8-fee21e72ec23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:15.290350 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75d8df-0c65-4ca2-a4d8-fee21e72ec23 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:15.290777 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75d8df-0c65-4ca2-a4d8-fee21e72ec23 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:15.291211 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75d8df-0c65-4ca2-a4d8-fee21e72ec23 None admin] PATH_INFO: `/v3/projects/643972018eac4175bcd3d9778028e93a` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:15.292566 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d75d8df-0c65-4ca2-a4d8-fee21e72ec23 None admin] RBAC: Authorizing `identity:get_project(project_id=643972018eac4175bcd3d9778028e93a)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:15.294173 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d75d8df-0c65-4ca2-a4d8-fee21e72ec23 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:15.296008 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 102/201] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:44:15 2022] GET /identity/v3/projects/643972018eac4175bcd3d9778028e93a => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:15.296942 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d75d8df-0c65-4ca2-a4d8-fee21e72ec23 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:15.331498 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e06fab-636d-4874-8dab-1d400849c758 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:15.365957 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d81824-1303-4702-b4a1-3ae12d08f9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:15.366673 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d81824-1303-4702-b4a1-3ae12d08f9f6 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:15.366991 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d81824-1303-4702-b4a1-3ae12d08f9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:15.367952 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d81824-1303-4702-b4a1-3ae12d08f9f6 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:15.368807 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d81824-1303-4702-b4a1-3ae12d08f9f6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:15.369227 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d81824-1303-4702-b4a1-3ae12d08f9f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:15.370554 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d81824-1303-4702-b4a1-3ae12d08f9f6 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:15.378937 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 100/202] 10.222.0.23 () {56 vars in 1101 bytes} [Fri May 6 14:44:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:15.379837 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d81824-1303-4702-b4a1-3ae12d08f9f6 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:15.418041 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d75d8df-0c65-4ca2-a4d8-fee21e72ec23 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:15.460743 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232e2a8c-4d94-4c38-bca3-54e1c54f70b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:15.461922 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e2a8c-4d94-4c38-bca3-54e1c54f70b8 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:15.462208 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e2a8c-4d94-4c38-bca3-54e1c54f70b8 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:15.462486 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e2a8c-4d94-4c38-bca3-54e1c54f70b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:15.463485 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232e2a8c-4d94-4c38-bca3-54e1c54f70b8 None admin] RBAC: Adding query filter params (role.id=d328f8040f2247d083309d839d2ff947, scope.project.id=643972018eac4175bcd3d9778028e93a, user.id=b7c1d929e29d48d4bd5322d408301fd8) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:15.464090 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232e2a8c-4d94-4c38-bca3-54e1c54f70b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:15.465572 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232e2a8c-4d94-4c38-bca3-54e1c54f70b8 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:15.473294 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 103/203] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:44:15 2022] GET /identity/v3/role_assignments?user.id=b7c1d929e29d48d4bd5322d408301fd8&scope.project.id=643972018eac4175bcd3d9778028e93a&role.id=d328f8040f2247d083309d839d2ff947 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:15.474384 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232e2a8c-4d94-4c38-bca3-54e1c54f70b8 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:15.479671 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d81824-1303-4702-b4a1-3ae12d08f9f6 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:15.519153 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2050575-f6c4-4738-b68a-88ab8ecca220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:15.519839 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2050575-f6c4-4738-b68a-88ab8ecca220 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:15.520187 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2050575-f6c4-4738-b68a-88ab8ecca220 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:15.520536 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2050575-f6c4-4738-b68a-88ab8ecca220 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:15.521493 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2050575-f6c4-4738-b68a-88ab8ecca220 None admin] RBAC: Adding query filter params (role.id=6dcd35d1a83748759d6ef9f026077835, scope.project.id=643972018eac4175bcd3d9778028e93a, user.id=b7c1d929e29d48d4bd5322d408301fd8) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:15.521871 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2050575-f6c4-4738-b68a-88ab8ecca220 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:15.523402 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2050575-f6c4-4738-b68a-88ab8ecca220 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:15.530264 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 101/204] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:44:15 2022] GET /identity/v3/role_assignments?user.id=b7c1d929e29d48d4bd5322d408301fd8&scope.project.id=643972018eac4175bcd3d9778028e93a&role.id=6dcd35d1a83748759d6ef9f026077835 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:15.639081 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232e2a8c-4d94-4c38-bca3-54e1c54f70b8 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:15.689296 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d53586-af44-42fb-9905-6663c45b9209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:15.690203 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d53586-af44-42fb-9905-6663c45b9209 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:15.690485 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d53586-af44-42fb-9905-6663c45b9209 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:15.690759 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d53586-af44-42fb-9905-6663c45b9209 None admin] PATH_INFO: `/v3/users/cdcebaa21c554387a0c7293ad5033b28` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:15.692059 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d53586-af44-42fb-9905-6663c45b9209 None admin] RBAC: Authorizing `identity:get_user(user_id=cdcebaa21c554387a0c7293ad5033b28)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:15.693598 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d53586-af44-42fb-9905-6663c45b9209 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:15.696194 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 104/205] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:44:15 2022] GET /identity/v3/users/cdcebaa21c554387a0c7293ad5033b28 => generated 289 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:15.701733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2050575-f6c4-4738-b68a-88ab8ecca220 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:15.872909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2050575-f6c4-4738-b68a-88ab8ecca220 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:15.922753 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc82563-8801-4f30-80db-5653cf7c7f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:15.923413 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc82563-8801-4f30-80db-5653cf7c7f67 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:15.923698 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc82563-8801-4f30-80db-5653cf7c7f67 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:15.923977 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc82563-8801-4f30-80db-5653cf7c7f67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:16.016507 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc82563-8801-4f30-80db-5653cf7c7f67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:16.018044 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc82563-8801-4f30-80db-5653cf7c7f67 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:16.153315 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 102/206] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:44:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 06 14:44:16.158742 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d53586-af44-42fb-9905-6663c45b9209 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:16.281380 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d53586-af44-42fb-9905-6663c45b9209 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:16.327358 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b88cce-0a46-4714-ae52-52332c44333f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:16.328225 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b88cce-0a46-4714-ae52-52332c44333f None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:16.328512 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b88cce-0a46-4714-ae52-52332c44333f None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:16.328811 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b88cce-0a46-4714-ae52-52332c44333f None admin] PATH_INFO: `/v3/projects/92337f7cabcc48ff8ca007298d1c3d78` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:16.333311 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b88cce-0a46-4714-ae52-52332c44333f None admin] RBAC: Authorizing `identity:get_project(project_id=92337f7cabcc48ff8ca007298d1c3d78)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:16.334761 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b88cce-0a46-4714-ae52-52332c44333f None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:16.337589 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 105/207] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:44:16 2022] GET /identity/v3/projects/92337f7cabcc48ff8ca007298d1c3d78 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:16.343333 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc82563-8801-4f30-80db-5653cf7c7f67 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:16.525639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc82563-8801-4f30-80db-5653cf7c7f67 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:16.575160 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56729acb-a3da-477d-b6d6-1e7287cb3b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:16.575824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56729acb-a3da-477d-b6d6-1e7287cb3b69 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:16.576107 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56729acb-a3da-477d-b6d6-1e7287cb3b69 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:16.576382 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56729acb-a3da-477d-b6d6-1e7287cb3b69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:16.577258 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56729acb-a3da-477d-b6d6-1e7287cb3b69 None admin] RBAC: Adding query filter params (role.id=e5c0610b69d640e595dccba09f44b81e, scope.project.id=92337f7cabcc48ff8ca007298d1c3d78, user.id=cdcebaa21c554387a0c7293ad5033b28) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:16.577574 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56729acb-a3da-477d-b6d6-1e7287cb3b69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:16.579034 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56729acb-a3da-477d-b6d6-1e7287cb3b69 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:16.586517 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916120b-c1ad-4ae4-817f-96edc4f78912 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:16.586806 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916120b-c1ad-4ae4-817f-96edc4f78912 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:16.587089 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916120b-c1ad-4ae4-817f-96edc4f78912 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:16.588732 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 106/208] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:16.590185 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 103/209] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:44:16 2022] GET /identity/v3/role_assignments?user.id=cdcebaa21c554387a0c7293ad5033b28&scope.project.id=92337f7cabcc48ff8ca007298d1c3d78&role.id=e5c0610b69d640e595dccba09f44b81e => generated 608 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:16.595935 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e96fe04-a56e-47de-bdac-b3e315c47adc None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:16.596524 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e96fe04-a56e-47de-bdac-b3e315c47adc None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:16.597059 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e96fe04-a56e-47de-bdac-b3e315c47adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:16.614858 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-3e96fe04-a56e-47de-bdac-b3e315c47adc None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:16.616548 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-3e96fe04-a56e-47de-bdac-b3e315c47adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:16.645735 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0563b-698a-4da4-b04f-9d8255a0813c None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:16.646399 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0563b-698a-4da4-b04f-9d8255a0813c None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:16.646871 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0563b-698a-4da4-b04f-9d8255a0813c None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:16.649038 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 104/210] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:16.654936 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539e65d-d8ab-4cca-9e7e-2ca67e206961 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:16.655223 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539e65d-d8ab-4cca-9e7e-2ca67e206961 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:16.655517 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539e65d-d8ab-4cca-9e7e-2ca67e206961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:16.690207 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-8539e65d-d8ab-4cca-9e7e-2ca67e206961 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:16.691879 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-8539e65d-d8ab-4cca-9e7e-2ca67e206961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:16.736343 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 107/211] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:44:16.796959 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 105/212] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 06 14:44:16.800809 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e96fe04-a56e-47de-bdac-b3e315c47adc None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:16.820161 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539e65d-d8ab-4cca-9e7e-2ca67e206961 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:16.936455 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e96fe04-a56e-47de-bdac-b3e315c47adc None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:16.938738 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539e65d-d8ab-4cca-9e7e-2ca67e206961 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:16.972059 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c46612-bc7a-41d6-ba5b-ac76e06d01b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:16.973428 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c46612-bc7a-41d6-ba5b-ac76e06d01b8 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:16.973983 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c46612-bc7a-41d6-ba5b-ac76e06d01b8 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:16.974429 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c46612-bc7a-41d6-ba5b-ac76e06d01b8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:16.974551 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2684e52-7b67-4ec3-bde7-e0d2732761a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:16.975368 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2684e52-7b67-4ec3-bde7-e0d2732761a7 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:16.975723 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2684e52-7b67-4ec3-bde7-e0d2732761a7 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:16.976070 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2684e52-7b67-4ec3-bde7-e0d2732761a7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:16.976458 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c46612-bc7a-41d6-ba5b-ac76e06d01b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:16.978352 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c46612-bc7a-41d6-ba5b-ac76e06d01b8 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:16.980173 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 108/213] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:44:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:16.985668 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c46612-bc7a-41d6-ba5b-ac76e06d01b8 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:16.987707 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2684e52-7b67-4ec3-bde7-e0d2732761a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:16.995652 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-c2684e52-7b67-4ec3-bde7-e0d2732761a7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 06 14:44:16.995652 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:16.995652 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:16.995652 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:16.995652 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:16.995652 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:16.995652 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:16.995652 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:16.995652 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:16.995652 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:16.995652 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:16.995652 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:16.995652 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:16.995652 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:16.995652 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:16.995652 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:16.996572 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:16.996572 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:16.996572 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:16.996572 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:16.996572 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:16.996572 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:16.996572 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:16.996572 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:16.996572 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:16.996572 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:16.996572 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:16.996572 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:44:16.996572 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:16.996572 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:16.996572 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:16.996572 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:16.997489 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:16.997489 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:16.997489 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:16.997489 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:16.997489 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:16.997489 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:16.997489 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:16.997489 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:16.997489 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:16.997489 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:16.997489 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 06 14:44:16.997489 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:44:16.997489 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 106/214] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:44:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:44:17.002956 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2684e52-7b67-4ec3-bde7-e0d2732761a7 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:17.094002 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c46612-bc7a-41d6-ba5b-ac76e06d01b8 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:17.108590 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2684e52-7b67-4ec3-bde7-e0d2732761a7 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:17.128153 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0894b787-6eaf-40a3-b165-510a99a8718e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:17.129204 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894b787-6eaf-40a3-b165-510a99a8718e None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:17.129686 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894b787-6eaf-40a3-b165-510a99a8718e None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:17.130153 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894b787-6eaf-40a3-b165-510a99a8718e None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:17.131072 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0894b787-6eaf-40a3-b165-510a99a8718e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:17.133307 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0894b787-6eaf-40a3-b165-510a99a8718e None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:17.145685 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aaaa6f-27f3-4268-9f33-00fd85573a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:17.146209 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aaaa6f-27f3-4268-9f33-00fd85573a27 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:17.146481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aaaa6f-27f3-4268-9f33-00fd85573a27 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:17.146782 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aaaa6f-27f3-4268-9f33-00fd85573a27 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:17.147573 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aaaa6f-27f3-4268-9f33-00fd85573a27 None admin] RBAC: Adding query filter params (name=member) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:17.147881 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aaaa6f-27f3-4268-9f33-00fd85573a27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:17.150029 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aaaa6f-27f3-4268-9f33-00fd85573a27 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:17.161514 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 107/215] 10.222.0.23 () {56 vars in 1101 bytes} [Fri May 6 14:44:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:17.166394 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aaaa6f-27f3-4268-9f33-00fd85573a27 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:17.179142 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 109/216] 10.222.0.23 () {60 vars in 1132 bytes} [Fri May 6 14:44:16 2022] POST /identity/v3/users => generated 297 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 14:44:17.277059 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aaaa6f-27f3-4268-9f33-00fd85573a27 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:17.313235 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0f0c6f-0745-4e40-b591-9b06b085b8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:17.314121 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f0c6f-0745-4e40-b591-9b06b085b8af None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:17.314402 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f0c6f-0745-4e40-b591-9b06b085b8af None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:17.314674 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f0c6f-0745-4e40-b591-9b06b085b8af None admin] PATH_INFO: `/v3/users/8f144e48c3f246ec8954f36bc0624707` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:17.332826 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0f0c6f-0745-4e40-b591-9b06b085b8af None admin] RBAC: Authorizing `identity:get_user(user_id=8f144e48c3f246ec8954f36bc0624707)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:17.334642 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0f0c6f-0745-4e40-b591-9b06b085b8af None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:17.336905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 108/217] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:44:17 2022] GET /identity/v3/users/8f144e48c3f246ec8954f36bc0624707 => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:17.342782 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0894b787-6eaf-40a3-b165-510a99a8718e None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:17.448204 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0894b787-6eaf-40a3-b165-510a99a8718e None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:17.481616 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd09c27-28e0-4df5-acce-9881cc533312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:17.482619 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd09c27-28e0-4df5-acce-9881cc533312 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:17.483124 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd09c27-28e0-4df5-acce-9881cc533312 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:17.483596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd09c27-28e0-4df5-acce-9881cc533312 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:17.556789 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd09c27-28e0-4df5-acce-9881cc533312 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:17.558263 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd09c27-28e0-4df5-acce-9881cc533312 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:17.666380 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 110/218] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:44:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 06 14:44:17.671831 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0f0c6f-0745-4e40-b591-9b06b085b8af None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:17.810090 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0f0c6f-0745-4e40-b591-9b06b085b8af None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:17.859089 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d046891-b008-4c64-ab1b-e83666f1e2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:17.860384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d046891-b008-4c64-ab1b-e83666f1e2d8 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:17.860901 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d046891-b008-4c64-ab1b-e83666f1e2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:17.861406 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d046891-b008-4c64-ab1b-e83666f1e2d8 None admin] PATH_INFO: `/v3/projects/643972018eac4175bcd3d9778028e93a` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:17.865517 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d046891-b008-4c64-ab1b-e83666f1e2d8 None admin] RBAC: Authorizing `identity:get_project(project_id=643972018eac4175bcd3d9778028e93a)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:17.867373 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d046891-b008-4c64-ab1b-e83666f1e2d8 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:17.869886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 109/219] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:44:17 2022] GET /identity/v3/projects/643972018eac4175bcd3d9778028e93a => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:17.875881 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd09c27-28e0-4df5-acce-9881cc533312 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:18.012039 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd09c27-28e0-4df5-acce-9881cc533312 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:18.051823 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78723c22-ac5d-4dae-8de6-7711de330f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:18.052525 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78723c22-ac5d-4dae-8de6-7711de330f7d None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:18.052858 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78723c22-ac5d-4dae-8de6-7711de330f7d None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:18.053155 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78723c22-ac5d-4dae-8de6-7711de330f7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:18.054035 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78723c22-ac5d-4dae-8de6-7711de330f7d None admin] RBAC: Adding query filter params (role.id=5f76e2e820b64d14b893c679427f4bed, scope.project.id=643972018eac4175bcd3d9778028e93a, user.id=8f144e48c3f246ec8954f36bc0624707) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:18.054377 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78723c22-ac5d-4dae-8de6-7711de330f7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:18.056628 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78723c22-ac5d-4dae-8de6-7711de330f7d None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:18.069039 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 111/220] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:44:17 2022] GET /identity/v3/role_assignments?user.id=8f144e48c3f246ec8954f36bc0624707&scope.project.id=643972018eac4175bcd3d9778028e93a&role.id=5f76e2e820b64d14b893c679427f4bed => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:18.086448 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd2512-1c06-4bcd-9f0c-394546684c99 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:18.087105 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd2512-1c06-4bcd-9f0c-394546684c99 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:18.087579 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd2512-1c06-4bcd-9f0c-394546684c99 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:18.089581 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 110/221] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:18.094949 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806f640-6dbe-4b7a-97f6-1f8313cfc868 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:18.095239 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806f640-6dbe-4b7a-97f6-1f8313cfc868 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:18.095516 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806f640-6dbe-4b7a-97f6-1f8313cfc868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:18.121893 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-1806f640-6dbe-4b7a-97f6-1f8313cfc868 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:18.123132 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-1806f640-6dbe-4b7a-97f6-1f8313cfc868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:18.250268 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 112/222] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:44:18.275535 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd2512-1c06-4bcd-9f0c-394546684c99 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:18.350115 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858abf8-ea97-4a8f-97c3-1acddcd33932 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:18.350404 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858abf8-ea97-4a8f-97c3-1acddcd33932 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:18.350597 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858abf8-ea97-4a8f-97c3-1acddcd33932 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:18.353531 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 113/223] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:18.358603 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc108ea-ca4f-49d9-995a-a97e29eebd20 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:18.358992 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc108ea-ca4f-49d9-995a-a97e29eebd20 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:18.359420 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc108ea-ca4f-49d9-995a-a97e29eebd20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:18.387903 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-6cc108ea-ca4f-49d9-995a-a97e29eebd20 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:18.390373 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-6cc108ea-ca4f-49d9-995a-a97e29eebd20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:18.455156 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd2512-1c06-4bcd-9f0c-394546684c99 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:18.519321 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 114/224] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:44:18.521805 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81384891-3a3c-45c3-85de-fcb37d9f4048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:18.522609 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81384891-3a3c-45c3-85de-fcb37d9f4048 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:18.522886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81384891-3a3c-45c3-85de-fcb37d9f4048 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:18.523156 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81384891-3a3c-45c3-85de-fcb37d9f4048 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:18.534801 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81384891-3a3c-45c3-85de-fcb37d9f4048 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:18.543910 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-81384891-3a3c-45c3-85de-fcb37d9f4048 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 06 14:44:18.543910 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:18.543910 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:18.543910 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:18.543910 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:18.543910 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:18.543910 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:18.543910 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:18.543910 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:18.543910 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:18.543910 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:18.543910 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:18.543910 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:18.543910 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:18.543910 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:18.543910 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:18.544827 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:18.544827 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:18.544827 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:18.544827 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:18.544827 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:18.544827 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:18.544827 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:18.544827 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:18.544827 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:18.544827 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:18.544827 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:18.544827 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:44:18.544827 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:18.544827 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:18.544827 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:18.544827 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:18.545600 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:18.545600 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:18.545600 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:18.545600 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:18.545600 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:18.545600 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:18.545600 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:18.545600 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:18.545600 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:18.545600 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:18.545600 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 06 14:44:18.545600 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:44:18.545600 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 111/225] 10.222.0.23 () {56 vars in 1114 bytes} [Fri May 6 14:44:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:44:18.551000 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc108ea-ca4f-49d9-995a-a97e29eebd20 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:18.562385 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81384891-3a3c-45c3-85de-fcb37d9f4048 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:18.682604 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81384891-3a3c-45c3-85de-fcb37d9f4048 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:18.720245 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f86917-51f0-4b4c-a2fb-cee7c918aeea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:18.721068 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f86917-51f0-4b4c-a2fb-cee7c918aeea None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:18.721344 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f86917-51f0-4b4c-a2fb-cee7c918aeea None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:18.721613 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f86917-51f0-4b4c-a2fb-cee7c918aeea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:18.734846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f86917-51f0-4b4c-a2fb-cee7c918aeea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:18.743940 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-f1f86917-51f0-4b4c-a2fb-cee7c918aeea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:44:18.743940 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:18.743940 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:18.743940 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:18.743940 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:18.743940 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:18.743940 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:18.743940 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:18.743940 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:18.743940 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:18.743940 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:18.743940 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:18.743940 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:18.743940 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:18.743940 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:18.743940 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:18.744904 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:18.744904 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:18.744904 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:18.744904 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:18.744904 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:18.744904 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:18.744904 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:18.744904 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:18.744904 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:18.744904 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:18.744904 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:18.744904 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:44:18.744904 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:18.744904 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:18.744904 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:18.744904 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:18.745839 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:18.745839 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:18.745839 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:18.745839 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:18.745839 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:18.745839 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:18.745839 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:18.745839 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:18.745839 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:18.745839 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:18.745839 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:44:18.745839 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:44:18.745839 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 112/226] 10.222.0.23 () {56 vars in 1096 bytes} [Fri May 6 14:44:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:18.751233 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f86917-51f0-4b4c-a2fb-cee7c918aeea None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:18.764399 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc108ea-ca4f-49d9-995a-a97e29eebd20 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:18.853921 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978f973-d8a8-4147-b8f2-15e6f9de23b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:18.854962 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978f973-d8a8-4147-b8f2-15e6f9de23b5 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:18.855460 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978f973-d8a8-4147-b8f2-15e6f9de23b5 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:18.855925 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978f973-d8a8-4147-b8f2-15e6f9de23b5 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:18.856896 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7978f973-d8a8-4147-b8f2-15e6f9de23b5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:18.857392 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7978f973-d8a8-4147-b8f2-15e6f9de23b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:18.859079 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7978f973-d8a8-4147-b8f2-15e6f9de23b5 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:18.879874 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 115/227] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:44:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:18.898115 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978f973-d8a8-4147-b8f2-15e6f9de23b5 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:18.906061 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f86917-51f0-4b4c-a2fb-cee7c918aeea None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:18.948050 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b32c7f-91a8-4aa1-a9d9-ed872941619a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:18.948695 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b32c7f-91a8-4aa1-a9d9-ed872941619a None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:18.948979 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b32c7f-91a8-4aa1-a9d9-ed872941619a None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:18.949256 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b32c7f-91a8-4aa1-a9d9-ed872941619a None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:18.950001 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b32c7f-91a8-4aa1-a9d9-ed872941619a None admin] RBAC: Adding query filter params (name=admin) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:18.950308 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b32c7f-91a8-4aa1-a9d9-ed872941619a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:18.951787 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b32c7f-91a8-4aa1-a9d9-ed872941619a None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:18.963264 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 113/228] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:44:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:18.969137 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b32c7f-91a8-4aa1-a9d9-ed872941619a None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:19.116142 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978f973-d8a8-4147-b8f2-15e6f9de23b5 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:19.139839 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b32c7f-91a8-4aa1-a9d9-ed872941619a None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:19.191205 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2258919b-cd66-4560-bdab-57dc9a05bc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:19.192498 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2258919b-cd66-4560-bdab-57dc9a05bc19 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:19.193020 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2258919b-cd66-4560-bdab-57dc9a05bc19 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:19.193482 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2258919b-cd66-4560-bdab-57dc9a05bc19 None admin] PATH_INFO: `/v3/users/b7c1d929e29d48d4bd5322d408301fd8` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:19.195018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2258919b-cd66-4560-bdab-57dc9a05bc19 None admin] RBAC: Authorizing `identity:get_user(user_id=b7c1d929e29d48d4bd5322d408301fd8)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:19.196835 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2258919b-cd66-4560-bdab-57dc9a05bc19 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:19.199262 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 114/229] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:44:18 2022] GET /identity/v3/users/b7c1d929e29d48d4bd5322d408301fd8 => generated 283 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:19.204333 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2258919b-cd66-4560-bdab-57dc9a05bc19 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:19.225923 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e7d515-3a35-4c0d-a4fc-ba6a110eafcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:19.227394 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e7d515-3a35-4c0d-a4fc-ba6a110eafcc None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:19.227696 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e7d515-3a35-4c0d-a4fc-ba6a110eafcc None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:19.227987 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e7d515-3a35-4c0d-a4fc-ba6a110eafcc None admin] PATH_INFO: `/v3/users/b7c1d929e29d48d4bd5322d408301fd8` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:19.233483 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e7d515-3a35-4c0d-a4fc-ba6a110eafcc None admin] RBAC: Authorizing `identity:get_user(user_id=b7c1d929e29d48d4bd5322d408301fd8)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:19.234921 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e7d515-3a35-4c0d-a4fc-ba6a110eafcc None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:19.236701 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 116/230] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:44:18 2022] GET /identity/v3/users/b7c1d929e29d48d4bd5322d408301fd8 => generated 283 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:19.242301 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e7d515-3a35-4c0d-a4fc-ba6a110eafcc None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:19.321744 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2258919b-cd66-4560-bdab-57dc9a05bc19 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:19.361954 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080fe0c2-2ac1-4ef0-ba0c-023859761577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:19.363165 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080fe0c2-2ac1-4ef0-ba0c-023859761577 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:19.363659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080fe0c2-2ac1-4ef0-ba0c-023859761577 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:19.364142 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080fe0c2-2ac1-4ef0-ba0c-023859761577 None admin] PATH_INFO: `/v3/projects/643972018eac4175bcd3d9778028e93a` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:19.365967 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080fe0c2-2ac1-4ef0-ba0c-023859761577 None admin] RBAC: Authorizing `identity:get_project(project_id=643972018eac4175bcd3d9778028e93a)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:19.367892 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080fe0c2-2ac1-4ef0-ba0c-023859761577 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:19.370747 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 115/231] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:44:19 2022] GET /identity/v3/projects/643972018eac4175bcd3d9778028e93a => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:19.378223 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080fe0c2-2ac1-4ef0-ba0c-023859761577 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:19.462150 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e7d515-3a35-4c0d-a4fc-ba6a110eafcc None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:19.534873 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76adff9f-3e54-4f5b-ae69-eab7246d8942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:19.535746 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76adff9f-3e54-4f5b-ae69-eab7246d8942 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:19.536027 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76adff9f-3e54-4f5b-ae69-eab7246d8942 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:19.536300 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76adff9f-3e54-4f5b-ae69-eab7246d8942 None admin] PATH_INFO: `/v3/projects/643972018eac4175bcd3d9778028e93a` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:19.538199 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76adff9f-3e54-4f5b-ae69-eab7246d8942 None admin] RBAC: Authorizing `identity:get_project(project_id=643972018eac4175bcd3d9778028e93a)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:19.539850 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76adff9f-3e54-4f5b-ae69-eab7246d8942 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:19.541588 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 117/232] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:44:19 2022] GET /identity/v3/projects/643972018eac4175bcd3d9778028e93a => generated 302 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:19.549632 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76adff9f-3e54-4f5b-ae69-eab7246d8942 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:19.590104 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080fe0c2-2ac1-4ef0-ba0c-023859761577 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:19.659385 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76adff9f-3e54-4f5b-ae69-eab7246d8942 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:19.666400 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a705fa0-b6e8-4eba-8c1e-1cdaeed0b073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:19.667313 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a705fa0-b6e8-4eba-8c1e-1cdaeed0b073 None admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:19.667591 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a705fa0-b6e8-4eba-8c1e-1cdaeed0b073 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:19.667867 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a705fa0-b6e8-4eba-8c1e-1cdaeed0b073 None admin] PATH_INFO: `/v3/projects/643972018eac4175bcd3d9778028e93a/users/b7c1d929e29d48d4bd5322d408301fd8/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:19.669893 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a705fa0-b6e8-4eba-8c1e-1cdaeed0b073 None admin] RBAC: Authorizing `identity:create_grant(project_id=643972018eac4175bcd3d9778028e93a, user_id=b7c1d929e29d48d4bd5322d408301fd8, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:19.671502 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a705fa0-b6e8-4eba-8c1e-1cdaeed0b073 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:19.696192 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 116/233] 10.222.0.23 () {58 vars in 1439 bytes} [Fri May 6 14:44:19 2022] PUT /identity/v3/projects/643972018eac4175bcd3d9778028e93a/users/b7c1d929e29d48d4bd5322d408301fd8/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:44:19.710208 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57839b44-f926-4189-83e7-31272380907b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:19.711225 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57839b44-f926-4189-83e7-31272380907b None admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:19.711872 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57839b44-f926-4189-83e7-31272380907b None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:19.711872 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57839b44-f926-4189-83e7-31272380907b None admin] PATH_INFO: `/v3/projects/643972018eac4175bcd3d9778028e93a/users/b7c1d929e29d48d4bd5322d408301fd8/roles/d328f8040f2247d083309d839d2ff947` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:19.717131 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57839b44-f926-4189-83e7-31272380907b None admin] RBAC: Authorizing `identity:create_grant(project_id=643972018eac4175bcd3d9778028e93a, user_id=b7c1d929e29d48d4bd5322d408301fd8, role_id=d328f8040f2247d083309d839d2ff947)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:19.718910 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57839b44-f926-4189-83e7-31272380907b None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:19.744876 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 118/234] 10.222.0.23 () {58 vars in 1439 bytes} [Fri May 6 14:44:19 2022] PUT /identity/v3/projects/643972018eac4175bcd3d9778028e93a/users/b7c1d929e29d48d4bd5322d408301fd8/roles/d328f8040f2247d083309d839d2ff947 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:44:20.112346 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678908b-a27c-47cb-83e9-5e5ff2e7e6f2 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:20.112946 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678908b-a27c-47cb-83e9-5e5ff2e7e6f2 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:20.113469 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678908b-a27c-47cb-83e9-5e5ff2e7e6f2 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:20.115218 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 117/235] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:20.122089 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6a187-bf46-4fdf-8011-d075ae77880d None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:20.122384 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6a187-bf46-4fdf-8011-d075ae77880d None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:20.123761 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6a187-bf46-4fdf-8011-d075ae77880d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:20.149971 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-90d6a187-bf46-4fdf-8011-d075ae77880d None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:20.149971 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-90d6a187-bf46-4fdf-8011-d075ae77880d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:20.270405 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 119/236] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:44:20.314293 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d678908b-a27c-47cb-83e9-5e5ff2e7e6f2 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:20.352483 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b3f49-bc95-4ea5-90f7-8d83b26a29cd None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:20.354195 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b3f49-bc95-4ea5-90f7-8d83b26a29cd None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:20.354512 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b3f49-bc95-4ea5-90f7-8d83b26a29cd None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:20.355909 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 120/237] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:44:20.372582 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9eb238-720e-4459-9666-3bd777158d84 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:20.372894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9eb238-720e-4459-9666-3bd777158d84 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:20.373088 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9eb238-720e-4459-9666-3bd777158d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:20.409815 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-af9eb238-720e-4459-9666-3bd777158d84 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:20.411013 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-af9eb238-720e-4459-9666-3bd777158d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:20.454825 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d678908b-a27c-47cb-83e9-5e5ff2e7e6f2 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:20.503618 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95946cd-bf33-4a01-afde-c2ff093e2213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:20.504789 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95946cd-bf33-4a01-afde-c2ff093e2213 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:20.506516 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95946cd-bf33-4a01-afde-c2ff093e2213 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:20.507364 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95946cd-bf33-4a01-afde-c2ff093e2213 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:20.511703 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95946cd-bf33-4a01-afde-c2ff093e2213 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:20.513570 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95946cd-bf33-4a01-afde-c2ff093e2213 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:20.516738 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 118/238] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:44:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:20.525047 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95946cd-bf33-4a01-afde-c2ff093e2213 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:20.554625 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 121/239] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 06 14:44:20.597834 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9eb238-720e-4459-9666-3bd777158d84 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:20.659279 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95946cd-bf33-4a01-afde-c2ff093e2213 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:20.702621 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c07279-f79d-4188-8725-de8bfb453767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:20.703306 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c07279-f79d-4188-8725-de8bfb453767 None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:20.703680 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c07279-f79d-4188-8725-de8bfb453767 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:20.703962 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c07279-f79d-4188-8725-de8bfb453767 None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:20.704733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c07279-f79d-4188-8725-de8bfb453767 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:20.706619 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c07279-f79d-4188-8725-de8bfb453767 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:20.737196 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 119/240] 10.222.0.23 () {60 vars in 1132 bytes} [Fri May 6 14:44:20 2022] POST /identity/v3/users => generated 293 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:44:20.783049 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9eb238-720e-4459-9666-3bd777158d84 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:20.822245 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3ac275-f6d2-4c60-853d-e6c262680824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:20.823073 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3ac275-f6d2-4c60-853d-e6c262680824 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:20.823361 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3ac275-f6d2-4c60-853d-e6c262680824 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:20.823643 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3ac275-f6d2-4c60-853d-e6c262680824 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:20.836395 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3ac275-f6d2-4c60-853d-e6c262680824 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:20.844422 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-0b3ac275-f6d2-4c60-853d-e6c262680824 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 06 14:44:20.844422 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:20.844422 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:20.844422 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:20.844422 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:20.844422 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:20.844422 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:20.844422 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:20.844422 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:20.844422 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:20.844422 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:20.844422 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:20.844422 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:20.844422 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:20.844422 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:20.844422 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:20.845394 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:20.845394 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:20.845394 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:20.845394 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:20.845394 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:20.845394 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:20.845394 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:20.845394 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:20.845394 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:20.845394 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:20.845394 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:20.845394 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:44:20.845394 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:20.845394 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:20.845394 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:20.845394 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:20.846181 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:20.846181 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:20.846181 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:20.846181 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:20.846181 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:20.846181 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:20.846181 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:20.846181 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:20.846181 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:20.846181 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:20.846181 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 06 14:44:20.846181 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:44:20.846181 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 122/241] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:44:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:44:20.854347 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c07279-f79d-4188-8725-de8bfb453767 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:20.946343 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139da25-241e-4334-8a07-9dc43db4b7c7 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:20.946658 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139da25-241e-4334-8a07-9dc43db4b7c7 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:20.946817 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139da25-241e-4334-8a07-9dc43db4b7c7 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:20.948457 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 123/242] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:44:20.965603 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc5150-967b-4681-a947-5ff2cb2c62f6 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:20.965922 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc5150-967b-4681-a947-5ff2cb2c62f6 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:20.966172 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc5150-967b-4681-a947-5ff2cb2c62f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:20.989563 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c07279-f79d-4188-8725-de8bfb453767 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:20.991679 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-d7cc5150-967b-4681-a947-5ff2cb2c62f6 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:21.001729 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-d7cc5150-967b-4681-a947-5ff2cb2c62f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:21.027688 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d314c4d-2060-4628-a6c7-dc7dcde58103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:21.028790 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d314c4d-2060-4628-a6c7-dc7dcde58103 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:21.029279 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d314c4d-2060-4628-a6c7-dc7dcde58103 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:21.030261 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d314c4d-2060-4628-a6c7-dc7dcde58103 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:21.031646 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d314c4d-2060-4628-a6c7-dc7dcde58103 None admin] RBAC: Adding query filter params (name=reader) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:21.032188 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d314c4d-2060-4628-a6c7-dc7dcde58103 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:21.033991 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d314c4d-2060-4628-a6c7-dc7dcde58103 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:21.053228 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 120/243] 10.222.0.23 () {56 vars in 1101 bytes} [Fri May 6 14:44:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:21.053228 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d314c4d-2060-4628-a6c7-dc7dcde58103 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:21.114687 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 124/244] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 06 14:44:21.141663 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cc5150-967b-4681-a947-5ff2cb2c62f6 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:21.186871 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d314c4d-2060-4628-a6c7-dc7dcde58103 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:21.225288 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a80bc-5a66-4c2b-b2db-12330ab8abef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:21.226094 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a80bc-5a66-4c2b-b2db-12330ab8abef None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:21.226405 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a80bc-5a66-4c2b-b2db-12330ab8abef None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:21.226680 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a80bc-5a66-4c2b-b2db-12330ab8abef None admin] PATH_INFO: `/v3/users/1257f345ccf84d32980f775c05c3c3d2` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:21.244843 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0a80bc-5a66-4c2b-b2db-12330ab8abef None admin] RBAC: Authorizing `identity:get_user(user_id=1257f345ccf84d32980f775c05c3c3d2)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:21.247042 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0a80bc-5a66-4c2b-b2db-12330ab8abef None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:21.249489 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 121/245] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:44:21 2022] GET /identity/v3/users/1257f345ccf84d32980f775c05c3c3d2 => generated 297 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:21.260924 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a80bc-5a66-4c2b-b2db-12330ab8abef None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:21.270277 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cc5150-967b-4681-a947-5ff2cb2c62f6 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:21.312661 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f5f800-b86f-4d2d-9b43-a8c5ccc060b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:21.313463 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5f800-b86f-4d2d-9b43-a8c5ccc060b1 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:21.313743 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5f800-b86f-4d2d-9b43-a8c5ccc060b1 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:21.314015 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5f800-b86f-4d2d-9b43-a8c5ccc060b1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:21.326327 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f5f800-b86f-4d2d-9b43-a8c5ccc060b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:21.336555 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-52f5f800-b86f-4d2d-9b43-a8c5ccc060b1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 06 14:44:21.336555 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:21.336555 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:21.336555 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:21.336555 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:21.336555 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:21.336555 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:21.336555 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:21.336555 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:21.336555 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:21.336555 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:21.336555 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:21.336555 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:21.336555 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:21.336555 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:21.336555 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:21.337485 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:21.337485 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:21.337485 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:21.337485 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:21.337485 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:21.337485 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:21.337485 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:21.337485 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:21.337485 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:21.337485 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:21.337485 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:21.337485 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:44:21.337485 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:21.337485 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:21.337485 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:21.337485 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:21.338268 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:21.338268 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:21.338268 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:21.338268 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:21.338268 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:21.338268 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:21.338268 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:21.338268 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:21.338268 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:21.338268 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:21.338268 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 06 14:44:21.338268 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:44:21.338857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 125/246] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:44:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:44:21.347937 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f5f800-b86f-4d2d-9b43-a8c5ccc060b1 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:21.394836 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a80bc-5a66-4c2b-b2db-12330ab8abef None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:21.446257 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddfe909-e174-4ca4-b6ea-11b0b313d13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:21.446923 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddfe909-e174-4ca4-b6ea-11b0b313d13c None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:21.447718 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddfe909-e174-4ca4-b6ea-11b0b313d13c None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:21.448001 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddfe909-e174-4ca4-b6ea-11b0b313d13c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:21.496030 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f5f800-b86f-4d2d-9b43-a8c5ccc060b1 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:21.534878 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddfe909-e174-4ca4-b6ea-11b0b313d13c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:21.536426 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddfe909-e174-4ca4-b6ea-11b0b313d13c None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:21.540857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d3d4c5-f728-4ec7-b383-70626821d661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:21.541771 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3d4c5-f728-4ec7-b383-70626821d661 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:21.542240 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3d4c5-f728-4ec7-b383-70626821d661 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:21.542623 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3d4c5-f728-4ec7-b383-70626821d661 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:21.543630 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d3d4c5-f728-4ec7-b383-70626821d661 None admin] RBAC: Adding query filter params (name=member) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:21.543962 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d3d4c5-f728-4ec7-b383-70626821d661 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:21.545841 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d3d4c5-f728-4ec7-b383-70626821d661 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:21.557129 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 126/247] 10.222.0.23 () {56 vars in 1101 bytes} [Fri May 6 14:44:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:21.564453 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d3d4c5-f728-4ec7-b383-70626821d661 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:21.700943 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 122/248] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:44:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 06 14:44:21.719537 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d3d4c5-f728-4ec7-b383-70626821d661 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:21.722280 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddfe909-e174-4ca4-b6ea-11b0b313d13c None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:21.764814 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0388a7-b5de-4829-849d-67b899fe69d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:21.764814 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0388a7-b5de-4829-849d-67b899fe69d1 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:21.764814 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0388a7-b5de-4829-849d-67b899fe69d1 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:21.764814 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0388a7-b5de-4829-849d-67b899fe69d1 None admin] PATH_INFO: `/v3/users/8f144e48c3f246ec8954f36bc0624707` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:21.765350 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0388a7-b5de-4829-849d-67b899fe69d1 None admin] RBAC: Authorizing `identity:get_user(user_id=8f144e48c3f246ec8954f36bc0624707)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:21.766795 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0388a7-b5de-4829-849d-67b899fe69d1 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:21.768735 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 127/249] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:44:21 2022] GET /identity/v3/users/8f144e48c3f246ec8954f36bc0624707 => generated 297 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:21.784664 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0388a7-b5de-4829-849d-67b899fe69d1 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:21.871786 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddfe909-e174-4ca4-b6ea-11b0b313d13c None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:21.903430 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0388a7-b5de-4829-849d-67b899fe69d1 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:21.916915 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f919a6c-5b83-440c-a580-180b9818ba14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:21.918103 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f919a6c-5b83-440c-a580-180b9818ba14 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:21.918586 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f919a6c-5b83-440c-a580-180b9818ba14 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:21.919103 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f919a6c-5b83-440c-a580-180b9818ba14 None admin] PATH_INFO: `/v3/projects/643972018eac4175bcd3d9778028e93a` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:21.920492 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f919a6c-5b83-440c-a580-180b9818ba14 None admin] RBAC: Authorizing `identity:get_project(project_id=643972018eac4175bcd3d9778028e93a)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:21.922154 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f919a6c-5b83-440c-a580-180b9818ba14 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:21.924378 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 123/250] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:44:21 2022] GET /identity/v3/projects/643972018eac4175bcd3d9778028e93a => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:21.930866 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f919a6c-5b83-440c-a580-180b9818ba14 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:21.938428 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b10ccc-1498-41c5-9a2f-a91e9784cb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:21.939293 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b10ccc-1498-41c5-9a2f-a91e9784cb67 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:21.939579 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b10ccc-1498-41c5-9a2f-a91e9784cb67 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:21.939851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b10ccc-1498-41c5-9a2f-a91e9784cb67 None admin] PATH_INFO: `/v3/projects/643972018eac4175bcd3d9778028e93a` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:21.941223 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b10ccc-1498-41c5-9a2f-a91e9784cb67 None admin] RBAC: Authorizing `identity:get_project(project_id=643972018eac4175bcd3d9778028e93a)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:21.942602 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b10ccc-1498-41c5-9a2f-a91e9784cb67 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:21.944643 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 128/251] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:44:21 2022] GET /identity/v3/projects/643972018eac4175bcd3d9778028e93a => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:21.954814 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b10ccc-1498-41c5-9a2f-a91e9784cb67 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:22.047095 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f919a6c-5b83-440c-a580-180b9818ba14 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:22.069369 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b10ccc-1498-41c5-9a2f-a91e9784cb67 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:22.099221 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33fe244-af6d-4495-bf84-994a392fa7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:22.099890 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33fe244-af6d-4495-bf84-994a392fa7fe None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:22.100169 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33fe244-af6d-4495-bf84-994a392fa7fe None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:22.100446 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33fe244-af6d-4495-bf84-994a392fa7fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:22.101339 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33fe244-af6d-4495-bf84-994a392fa7fe None admin] RBAC: Adding query filter params (role.id=e5c0610b69d640e595dccba09f44b81e, scope.project.id=643972018eac4175bcd3d9778028e93a, user.id=1257f345ccf84d32980f775c05c3c3d2) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:22.102186 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33fe244-af6d-4495-bf84-994a392fa7fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:22.105383 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33fe244-af6d-4495-bf84-994a392fa7fe None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:22.114502 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 124/252] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:44:21 2022] GET /identity/v3/role_assignments?user.id=1257f345ccf84d32980f775c05c3c3d2&scope.project.id=643972018eac4175bcd3d9778028e93a&role.id=e5c0610b69d640e595dccba09f44b81e => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:22.124727 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71359ecf-f6d8-4a9d-856b-63b641b990e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:22.125631 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71359ecf-f6d8-4a9d-856b-63b641b990e4 None admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:22.125909 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71359ecf-f6d8-4a9d-856b-63b641b990e4 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:22.126179 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71359ecf-f6d8-4a9d-856b-63b641b990e4 None admin] PATH_INFO: `/v3/projects/643972018eac4175bcd3d9778028e93a/users/8f144e48c3f246ec8954f36bc0624707/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:22.128481 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71359ecf-f6d8-4a9d-856b-63b641b990e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=643972018eac4175bcd3d9778028e93a, user_id=8f144e48c3f246ec8954f36bc0624707, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:22.129965 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71359ecf-f6d8-4a9d-856b-63b641b990e4 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:22.152918 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 129/253] 10.222.0.23 () {58 vars in 1439 bytes} [Fri May 6 14:44:21 2022] PUT /identity/v3/projects/643972018eac4175bcd3d9778028e93a/users/8f144e48c3f246ec8954f36bc0624707/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:44:22.584334 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff46641-4ff0-45f3-9b65-02f36b346d78 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:22.584905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff46641-4ff0-45f3-9b65-02f36b346d78 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:22.584905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff46641-4ff0-45f3-9b65-02f36b346d78 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:22.586697 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 125/254] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:22.593918 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9266aa8-a9e5-4150-b916-0c91da7ddc26 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:22.594379 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9266aa8-a9e5-4150-b916-0c91da7ddc26 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:22.594454 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9266aa8-a9e5-4150-b916-0c91da7ddc26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:22.613448 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-a9266aa8-a9e5-4150-b916-0c91da7ddc26 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:22.614652 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-a9266aa8-a9e5-4150-b916-0c91da7ddc26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:22.700159 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 130/255] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:44:22.735812 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff46641-4ff0-45f3-9b65-02f36b346d78 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:22.861439 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff46641-4ff0-45f3-9b65-02f36b346d78 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:22.886328 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a36c4-7f8f-46eb-9bd0-9e7934e6763b None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:22.886657 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a36c4-7f8f-46eb-9bd0-9e7934e6763b None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:22.886901 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a36c4-7f8f-46eb-9bd0-9e7934e6763b None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:22.888519 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 131/256] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:44:22.893472 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5a410-bdea-4754-945f-78d2a6bf9354 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:22.893761 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5a410-bdea-4754-945f-78d2a6bf9354 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:22.894043 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5a410-bdea-4754-945f-78d2a6bf9354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:22.912996 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-b9a5a410-bdea-4754-945f-78d2a6bf9354 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:22.914701 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-b9a5a410-bdea-4754-945f-78d2a6bf9354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:22.918064 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fccca59-bd8f-4b12-8f7d-e0ab8aa6fb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:22.918872 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fccca59-bd8f-4b12-8f7d-e0ab8aa6fb6e None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:22.919153 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fccca59-bd8f-4b12-8f7d-e0ab8aa6fb6e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:22.919428 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fccca59-bd8f-4b12-8f7d-e0ab8aa6fb6e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:22.936170 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fccca59-bd8f-4b12-8f7d-e0ab8aa6fb6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:22.943868 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-7fccca59-bd8f-4b12-8f7d-e0ab8aa6fb6e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:44:22.943868 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:22.943868 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:22.943868 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:22.943868 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:22.943868 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:22.943868 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:22.943868 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:22.943868 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:22.943868 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:22.943868 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:22.943868 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:22.943868 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:22.943868 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:22.943868 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:22.943868 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:22.944797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:22.944797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:22.944797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:22.944797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:22.944797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:22.944797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:22.944797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:22.944797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:22.944797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:22.944797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:22.944797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:22.944797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:44:22.944797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:22.944797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:22.944797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:22.944797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:22.945665 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:22.945665 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:22.945665 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:22.945665 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:22.945665 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:22.945665 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:22.945665 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:22.945665 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:22.945665 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:22.945665 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:22.945665 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:44:22.945665 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:44:22.945665 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 126/257] 10.222.0.23 () {56 vars in 1096 bytes} [Fri May 6 14:44:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:22.958389 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fccca59-bd8f-4b12-8f7d-e0ab8aa6fb6e None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:23.058944 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 132/258] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 06 14:44:23.097433 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a5a410-bdea-4754-945f-78d2a6bf9354 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:23.158751 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fccca59-bd8f-4b12-8f7d-e0ab8aa6fb6e None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:23.218697 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02fe99f-2d17-461f-a46f-c7a939fc9f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:23.219369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02fe99f-2d17-461f-a46f-c7a939fc9f5a None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:23.219652 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02fe99f-2d17-461f-a46f-c7a939fc9f5a None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:23.219930 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02fe99f-2d17-461f-a46f-c7a939fc9f5a None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:23.220707 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02fe99f-2d17-461f-a46f-c7a939fc9f5a None admin] RBAC: Adding query filter params (name=admin) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:23.221017 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02fe99f-2d17-461f-a46f-c7a939fc9f5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:23.222411 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02fe99f-2d17-461f-a46f-c7a939fc9f5a None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:23.240563 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 127/259] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:44:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:23.246971 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02fe99f-2d17-461f-a46f-c7a939fc9f5a None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:23.278747 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a5a410-bdea-4754-945f-78d2a6bf9354 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:23.347727 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d204423-845c-407b-898b-f06c1b75dcb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:23.348681 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d204423-845c-407b-898b-f06c1b75dcb1 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:23.348969 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d204423-845c-407b-898b-f06c1b75dcb1 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:23.349244 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d204423-845c-407b-898b-f06c1b75dcb1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:23.372501 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d204423-845c-407b-898b-f06c1b75dcb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:23.390814 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-1d204423-845c-407b-898b-f06c1b75dcb1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 06 14:44:23.390814 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:23.390814 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:23.390814 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:23.390814 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:23.390814 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:23.390814 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:23.390814 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:23.390814 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:23.390814 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:23.390814 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:23.390814 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:23.390814 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:23.390814 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:23.390814 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:23.390814 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:23.391743 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:23.391743 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:23.391743 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:23.391743 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:23.391743 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:23.391743 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:23.391743 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:23.391743 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:23.391743 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:23.391743 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:23.391743 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:23.391743 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:44:23.391743 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:23.391743 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:23.391743 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:23.391743 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:23.392532 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:23.392532 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:23.392532 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:23.392532 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:23.392532 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:23.392532 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:23.392532 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:23.392532 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:23.392532 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:23.392532 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:23.392532 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 06 14:44:23.392532 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:44:23.392532 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 133/260] 10.222.0.23 () {56 vars in 1114 bytes} [Fri May 6 14:44:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:44:23.412905 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d204423-845c-407b-898b-f06c1b75dcb1 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:23.432319 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02fe99f-2d17-461f-a46f-c7a939fc9f5a None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:23.500266 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a70977-5135-466e-8bea-4772879b8667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:23.501468 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a70977-5135-466e-8bea-4772879b8667 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:23.501930 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a70977-5135-466e-8bea-4772879b8667 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:23.502378 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a70977-5135-466e-8bea-4772879b8667 None admin] PATH_INFO: `/v3/users/b7c1d929e29d48d4bd5322d408301fd8` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:23.504634 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a70977-5135-466e-8bea-4772879b8667 None admin] RBAC: Authorizing `identity:get_user(user_id=b7c1d929e29d48d4bd5322d408301fd8)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:23.506420 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a70977-5135-466e-8bea-4772879b8667 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:23.512963 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 128/261] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:44:23 2022] GET /identity/v3/users/b7c1d929e29d48d4bd5322d408301fd8 => generated 283 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:23.518055 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a70977-5135-466e-8bea-4772879b8667 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:23.565766 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d204423-845c-407b-898b-f06c1b75dcb1 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:23.604051 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3953ae89-d886-412d-9481-c2c3213a55d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:23.605836 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953ae89-d886-412d-9481-c2c3213a55d9 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:23.606303 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953ae89-d886-412d-9481-c2c3213a55d9 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:23.606759 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953ae89-d886-412d-9481-c2c3213a55d9 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:23.607878 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3953ae89-d886-412d-9481-c2c3213a55d9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:23.608364 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3953ae89-d886-412d-9481-c2c3213a55d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:23.610385 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3953ae89-d886-412d-9481-c2c3213a55d9 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:23.620188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 134/262] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:44:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:23.628646 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3953ae89-d886-412d-9481-c2c3213a55d9 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:23.697839 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a70977-5135-466e-8bea-4772879b8667 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:23.737405 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d01d89e-0aab-46df-9e76-e257a71cd88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:23.738058 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01d89e-0aab-46df-9e76-e257a71cd88d None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:23.738338 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01d89e-0aab-46df-9e76-e257a71cd88d None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:23.738610 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01d89e-0aab-46df-9e76-e257a71cd88d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:23.840152 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d01d89e-0aab-46df-9e76-e257a71cd88d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:23.840345 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3953ae89-d886-412d-9481-c2c3213a55d9 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:23.841649 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d01d89e-0aab-46df-9e76-e257a71cd88d None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:23.926200 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f61dd-69c2-4ac7-9ce5-cd68b94cf82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:23.927090 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f61dd-69c2-4ac7-9ce5-cd68b94cf82b None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:23.927377 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f61dd-69c2-4ac7-9ce5-cd68b94cf82b None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:23.927651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f61dd-69c2-4ac7-9ce5-cd68b94cf82b None admin] PATH_INFO: `/v3/users/b7c1d929e29d48d4bd5322d408301fd8` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:23.928881 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5f61dd-69c2-4ac7-9ce5-cd68b94cf82b None admin] RBAC: Authorizing `identity:get_user(user_id=b7c1d929e29d48d4bd5322d408301fd8)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:23.931535 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5f61dd-69c2-4ac7-9ce5-cd68b94cf82b None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:23.933516 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 135/263] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:44:23 2022] GET /identity/v3/users/b7c1d929e29d48d4bd5322d408301fd8 => generated 283 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:23.955370 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 129/264] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:44:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 06 14:44:23.956895 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f61dd-69c2-4ac7-9ce5-cd68b94cf82b None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:23.957033 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d01d89e-0aab-46df-9e76-e257a71cd88d None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:24.057947 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d01d89e-0aab-46df-9e76-e257a71cd88d None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:24.087033 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f61dd-69c2-4ac7-9ce5-cd68b94cf82b None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:24.093066 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca8602-f77c-4084-9f14-121444da4a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:24.093761 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca8602-f77c-4084-9f14-121444da4a24 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:24.094051 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca8602-f77c-4084-9f14-121444da4a24 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:24.094327 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca8602-f77c-4084-9f14-121444da4a24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:24.133337 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ea47d-0439-4051-9d39-0f1001aa919c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:24.134222 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ea47d-0439-4051-9d39-0f1001aa919c None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:24.134504 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ea47d-0439-4051-9d39-0f1001aa919c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:24.134776 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ea47d-0439-4051-9d39-0f1001aa919c None admin] PATH_INFO: `/v3/projects/643972018eac4175bcd3d9778028e93a` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:24.136009 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20ea47d-0439-4051-9d39-0f1001aa919c None admin] RBAC: Authorizing `identity:get_project(project_id=643972018eac4175bcd3d9778028e93a)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:24.137493 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20ea47d-0439-4051-9d39-0f1001aa919c None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:24.139493 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 136/265] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:44:23 2022] GET /identity/v3/projects/643972018eac4175bcd3d9778028e93a => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:24.166217 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ea47d-0439-4051-9d39-0f1001aa919c None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:24.169686 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ca8602-f77c-4084-9f14-121444da4a24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:24.171175 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ca8602-f77c-4084-9f14-121444da4a24 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:24.276742 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 130/266] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:44:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 06 14:44:24.279017 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca8602-f77c-4084-9f14-121444da4a24 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:24.380466 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca8602-f77c-4084-9f14-121444da4a24 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:24.405413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ea47d-0439-4051-9d39-0f1001aa919c None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:24.421798 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8579371a-e5ae-49c5-ae78-a1b4ab39af6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:24.422686 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8579371a-e5ae-49c5-ae78-a1b4ab39af6d None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:24.422971 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8579371a-e5ae-49c5-ae78-a1b4ab39af6d None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:24.423251 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8579371a-e5ae-49c5-ae78-a1b4ab39af6d None admin] PATH_INFO: `/v3/projects/643972018eac4175bcd3d9778028e93a` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:24.425449 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8579371a-e5ae-49c5-ae78-a1b4ab39af6d None admin] RBAC: Authorizing `identity:get_project(project_id=643972018eac4175bcd3d9778028e93a)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:24.432655 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8579371a-e5ae-49c5-ae78-a1b4ab39af6d None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:24.432655 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 131/267] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:44:24 2022] GET /identity/v3/projects/643972018eac4175bcd3d9778028e93a => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:24.437870 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8579371a-e5ae-49c5-ae78-a1b4ab39af6d None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:24.484872 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500af7f2-a1c7-4bd4-a953-5f58185350c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:24.485552 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500af7f2-a1c7-4bd4-a953-5f58185350c7 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:24.485841 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500af7f2-a1c7-4bd4-a953-5f58185350c7 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:24.486121 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500af7f2-a1c7-4bd4-a953-5f58185350c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:24.487031 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500af7f2-a1c7-4bd4-a953-5f58185350c7 None admin] RBAC: Adding query filter params (role.id=6dcd35d1a83748759d6ef9f026077835, scope.project.id=643972018eac4175bcd3d9778028e93a, user.id=b7c1d929e29d48d4bd5322d408301fd8) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:24.487348 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500af7f2-a1c7-4bd4-a953-5f58185350c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:24.488822 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500af7f2-a1c7-4bd4-a953-5f58185350c7 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:24.511116 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 137/268] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:44:24 2022] GET /identity/v3/role_assignments?user.id=b7c1d929e29d48d4bd5322d408301fd8&scope.project.id=643972018eac4175bcd3d9778028e93a&role.id=6dcd35d1a83748759d6ef9f026077835 => generated 608 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:24.512469 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64cf83-e4b2-4285-b6c3-831fbf785945 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:24.512786 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64cf83-e4b2-4285-b6c3-831fbf785945 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:24.513066 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64cf83-e4b2-4285-b6c3-831fbf785945 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:24.514555 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 138/269] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:24.522360 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de47ac-0b39-4ea0-9eab-b61664ee0d62 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:24.522673 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de47ac-0b39-4ea0-9eab-b61664ee0d62 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:24.522933 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de47ac-0b39-4ea0-9eab-b61664ee0d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:24.565849 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-18de47ac-0b39-4ea0-9eab-b61664ee0d62 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:24.567001 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-18de47ac-0b39-4ea0-9eab-b61664ee0d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:24.621643 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8579371a-e5ae-49c5-ae78-a1b4ab39af6d None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:24.701479 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a729ae3-a3d7-4d17-9eff-71e2c201891d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:24.703302 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a729ae3-a3d7-4d17-9eff-71e2c201891d None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:24.704025 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a729ae3-a3d7-4d17-9eff-71e2c201891d None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:24.704531 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a729ae3-a3d7-4d17-9eff-71e2c201891d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:24.706215 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a729ae3-a3d7-4d17-9eff-71e2c201891d None admin] RBAC: Adding query filter params (role.id=d328f8040f2247d083309d839d2ff947, scope.project.id=643972018eac4175bcd3d9778028e93a, user.id=b7c1d929e29d48d4bd5322d408301fd8) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:24.706755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a729ae3-a3d7-4d17-9eff-71e2c201891d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:24.708467 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a729ae3-a3d7-4d17-9eff-71e2c201891d None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:24.721461 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 132/270] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:44:24 2022] GET /identity/v3/role_assignments?user.id=b7c1d929e29d48d4bd5322d408301fd8&scope.project.id=643972018eac4175bcd3d9778028e93a&role.id=d328f8040f2247d083309d839d2ff947 => generated 608 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:24.734133 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 139/271] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:44:24.769664 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a729ae3-a3d7-4d17-9eff-71e2c201891d None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:24.915466 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a729ae3-a3d7-4d17-9eff-71e2c201891d None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:24.966618 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf4e452-3fdc-4c96-8e81-c4f3229ddf2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:24.967823 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf4e452-3fdc-4c96-8e81-c4f3229ddf2f None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:24.968327 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf4e452-3fdc-4c96-8e81-c4f3229ddf2f None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:24.968842 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf4e452-3fdc-4c96-8e81-c4f3229ddf2f None admin] PATH_INFO: `/v3/roles/member` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:24.981498 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf4e452-3fdc-4c96-8e81-c4f3229ddf2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:24.989649 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-4bf4e452-3fdc-4c96-8e81-c4f3229ddf2f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 06 14:44:24.989649 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:24.989649 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:24.989649 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:24.989649 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:24.989649 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:24.989649 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:24.989649 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:24.989649 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:24.989649 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:24.989649 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:24.989649 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:24.989649 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:24.989649 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:24.989649 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:24.989649 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:24.990516 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:24.990516 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:24.990516 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:24.990516 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:24.990516 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:24.990516 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:24.990516 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:24.990516 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:24.990516 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:24.990516 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:24.990516 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:24.990516 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:44:24.990516 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:24.990516 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:24.990516 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:24.990516 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:24.991308 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:24.991308 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:24.991308 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:24.991308 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:24.991308 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:24.991308 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:24.991308 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:24.991308 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:24.991308 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:24.991308 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:24.991308 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 06 14:44:24.991308 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:44:24.994069 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 133/272] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:44:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:24.999817 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18de47ac-0b39-4ea0-9eab-b61664ee0d62 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:25.111380 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18de47ac-0b39-4ea0-9eab-b61664ee0d62 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:25.147976 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7465f044-d8aa-47ad-abcb-e0afc9908172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:25.149604 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7465f044-d8aa-47ad-abcb-e0afc9908172 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:25.150090 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7465f044-d8aa-47ad-abcb-e0afc9908172 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:25.150624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7465f044-d8aa-47ad-abcb-e0afc9908172 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:25.151636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7465f044-d8aa-47ad-abcb-e0afc9908172 None admin] RBAC: Adding query filter params (name=member) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:25.152236 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7465f044-d8aa-47ad-abcb-e0afc9908172 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:25.153907 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7465f044-d8aa-47ad-abcb-e0afc9908172 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:25.167828 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 140/273] 10.222.0.23 () {56 vars in 1101 bytes} [Fri May 6 14:44:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:25.169226 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf4e452-3fdc-4c96-8e81-c4f3229ddf2f None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:25.286209 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf4e452-3fdc-4c96-8e81-c4f3229ddf2f None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:25.319638 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de795c-b999-42b9-9219-bc1616953bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:25.321742 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de795c-b999-42b9-9219-bc1616953bba None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:25.322235 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de795c-b999-42b9-9219-bc1616953bba None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:25.322709 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de795c-b999-42b9-9219-bc1616953bba None admin] PATH_INFO: `/v3/users/c93e3f6d8b514749bb8584fbf5282280` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:25.334162 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ef6ee-b9ef-476a-9c03-4a118cfe7e26 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:25.334458 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ef6ee-b9ef-476a-9c03-4a118cfe7e26 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:25.334733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ef6ee-b9ef-476a-9c03-4a118cfe7e26 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:25.337068 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 141/274] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:44:25.342356 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce2af7-28a5-419a-bb8a-79e1d9e98446 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:25.342643 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce2af7-28a5-419a-bb8a-79e1d9e98446 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:25.342951 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce2af7-28a5-419a-bb8a-79e1d9e98446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:25.344371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94de795c-b999-42b9-9219-bc1616953bba None admin] RBAC: Authorizing `identity:get_user(user_id=c93e3f6d8b514749bb8584fbf5282280)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:25.346090 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94de795c-b999-42b9-9219-bc1616953bba None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:25.348029 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 134/275] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:44:25 2022] GET /identity/v3/users/c93e3f6d8b514749bb8584fbf5282280 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:25.360837 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de795c-b999-42b9-9219-bc1616953bba None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:25.365425 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-92ce2af7-28a5-419a-bb8a-79e1d9e98446 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:25.366763 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-92ce2af7-28a5-419a-bb8a-79e1d9e98446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:25.440592 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 142/276] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 06 14:44:25.460573 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d29161-b181-460e-845b-4bd8b3d8006e None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:25.461613 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d29161-b181-460e-845b-4bd8b3d8006e None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:25.461947 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d29161-b181-460e-845b-4bd8b3d8006e None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:25.463637 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 143/277] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:25.468587 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44201b07-3532-4066-890a-68c7b34bc79e None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:25.468949 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44201b07-3532-4066-890a-68c7b34bc79e None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:25.469231 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44201b07-3532-4066-890a-68c7b34bc79e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:25.486985 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-44201b07-3532-4066-890a-68c7b34bc79e None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:25.487552 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de795c-b999-42b9-9219-bc1616953bba None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:25.488215 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-44201b07-3532-4066-890a-68c7b34bc79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:25.524658 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12c5d3a-5927-4838-8476-a74508fa37bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:25.525323 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c5d3a-5927-4838-8476-a74508fa37bf None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:25.525609 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c5d3a-5927-4838-8476-a74508fa37bf None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:25.525886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c5d3a-5927-4838-8476-a74508fa37bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:25.526715 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12c5d3a-5927-4838-8476-a74508fa37bf None admin] RBAC: Adding query filter params (role.id=5f76e2e820b64d14b893c679427f4bed, user.id=c93e3f6d8b514749bb8584fbf5282280, scope.system=all) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:25.527023 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12c5d3a-5927-4838-8476-a74508fa37bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:25.528471 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12c5d3a-5927-4838-8476-a74508fa37bf None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:25.533881 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 135/278] 10.222.0.23 () {56 vars in 1308 bytes} [Fri May 6 14:44:25 2022] GET /identity/v3/role_assignments?user.id=c93e3f6d8b514749bb8584fbf5282280&scope.system=all&role.id=5f76e2e820b64d14b893c679427f4bed => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:25.534677 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12c5d3a-5927-4838-8476-a74508fa37bf None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:25.569027 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 144/279] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 06 14:44:25.588929 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44201b07-3532-4066-890a-68c7b34bc79e None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:25.666840 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12c5d3a-5927-4838-8476-a74508fa37bf None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:25.704500 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb32c73-2e9e-4d59-8148-14e5682d3965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:25.705327 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb32c73-2e9e-4d59-8148-14e5682d3965 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:25.705556 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb32c73-2e9e-4d59-8148-14e5682d3965 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:25.705835 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb32c73-2e9e-4d59-8148-14e5682d3965 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:25.707181 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb32c73-2e9e-4d59-8148-14e5682d3965 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:25.708756 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb32c73-2e9e-4d59-8148-14e5682d3965 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:25.710709 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 136/280] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:44:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:25.716258 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb32c73-2e9e-4d59-8148-14e5682d3965 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:25.720585 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44201b07-3532-4066-890a-68c7b34bc79e None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:25.766893 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f45dd31-779b-435b-8243-a2bdbbbba923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:25.768252 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45dd31-779b-435b-8243-a2bdbbbba923 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:25.768927 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45dd31-779b-435b-8243-a2bdbbbba923 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:25.769220 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45dd31-779b-435b-8243-a2bdbbbba923 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:25.783708 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f45dd31-779b-435b-8243-a2bdbbbba923 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:25.794496 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-4f45dd31-779b-435b-8243-a2bdbbbba923 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 06 14:44:25.794496 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:25.794496 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:25.794496 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:25.794496 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:25.794496 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:25.794496 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:25.794496 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:25.794496 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:25.794496 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:25.794496 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:25.794496 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:25.794496 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:25.794496 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:25.794496 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:25.794496 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:25.795403 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:25.795403 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:25.795403 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:25.795403 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:25.795403 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:25.795403 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:25.795403 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:25.795403 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:25.795403 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:25.795403 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:25.795403 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:25.795403 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:44:25.795403 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:25.795403 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:25.795403 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:25.795403 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:25.796226 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:25.796226 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:25.796226 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:25.796226 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:25.796226 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:25.796226 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:25.796226 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:25.796226 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:25.796226 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:25.796226 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:25.796226 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 06 14:44:25.796226 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:44:25.798156 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 145/281] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:44:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:25.806349 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f45dd31-779b-435b-8243-a2bdbbbba923 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:25.833517 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb32c73-2e9e-4d59-8148-14e5682d3965 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:25.869048 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132a674b-8218-4b1f-b96e-9795dcf13e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:25.870077 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a674b-8218-4b1f-b96e-9795dcf13e35 None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:25.870565 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a674b-8218-4b1f-b96e-9795dcf13e35 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:25.871032 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a674b-8218-4b1f-b96e-9795dcf13e35 None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:25.871940 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132a674b-8218-4b1f-b96e-9795dcf13e35 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:25.873643 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132a674b-8218-4b1f-b96e-9795dcf13e35 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:25.899879 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 137/282] 10.222.0.23 () {60 vars in 1132 bytes} [Fri May 6 14:44:25 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 14:44:25.920888 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f45dd31-779b-435b-8243-a2bdbbbba923 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:25.921082 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed377da-9513-46dc-ba7b-53659209d160 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:25.921341 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed377da-9513-46dc-ba7b-53659209d160 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:25.921619 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed377da-9513-46dc-ba7b-53659209d160 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:25.923025 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 138/283] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:44:25.928800 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c982d58-fee1-40d1-ad17-58f4d80caca8 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:25.929191 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c982d58-fee1-40d1-ad17-58f4d80caca8 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:25.930526 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c982d58-fee1-40d1-ad17-58f4d80caca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:25.947927 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-4c982d58-fee1-40d1-ad17-58f4d80caca8 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:25.949353 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-4c982d58-fee1-40d1-ad17-58f4d80caca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:25.966285 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e124fa4a-fe51-43ad-ac93-cfbf1e20aa20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:25.967911 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e124fa4a-fe51-43ad-ac93-cfbf1e20aa20 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:25.968454 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e124fa4a-fe51-43ad-ac93-cfbf1e20aa20 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:25.968932 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e124fa4a-fe51-43ad-ac93-cfbf1e20aa20 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:25.969862 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e124fa4a-fe51-43ad-ac93-cfbf1e20aa20 None admin] RBAC: Adding query filter params (name=member) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:25.970323 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e124fa4a-fe51-43ad-ac93-cfbf1e20aa20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:25.972567 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e124fa4a-fe51-43ad-ac93-cfbf1e20aa20 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:25.985920 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 146/284] 10.222.0.23 () {56 vars in 1101 bytes} [Fri May 6 14:44:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:25.991157 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e124fa4a-fe51-43ad-ac93-cfbf1e20aa20 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:26.039634 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 139/285] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 06 14:44:26.063286 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c982d58-fee1-40d1-ad17-58f4d80caca8 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:26.127302 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e124fa4a-fe51-43ad-ac93-cfbf1e20aa20 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:26.169321 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ef212-33ee-4ecc-b82b-3d1686198340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:26.170209 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ef212-33ee-4ecc-b82b-3d1686198340 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:26.170641 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ef212-33ee-4ecc-b82b-3d1686198340 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:26.170810 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ef212-33ee-4ecc-b82b-3d1686198340 None admin] PATH_INFO: `/v3/users/8f144e48c3f246ec8954f36bc0624707` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:26.171962 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9ef212-33ee-4ecc-b82b-3d1686198340 None admin] RBAC: Authorizing `identity:get_user(user_id=8f144e48c3f246ec8954f36bc0624707)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:26.175052 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9ef212-33ee-4ecc-b82b-3d1686198340 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:26.175165 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 147/286] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:44:25 2022] GET /identity/v3/users/8f144e48c3f246ec8954f36bc0624707 => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:26.181679 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ef212-33ee-4ecc-b82b-3d1686198340 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:26.190081 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c982d58-fee1-40d1-ad17-58f4d80caca8 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:26.261359 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5dd376-9393-4bfc-8a2a-bc349a24fc05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:26.262425 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5dd376-9393-4bfc-8a2a-bc349a24fc05 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:26.262764 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5dd376-9393-4bfc-8a2a-bc349a24fc05 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:26.263043 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5dd376-9393-4bfc-8a2a-bc349a24fc05 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:26.275709 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5dd376-9393-4bfc-8a2a-bc349a24fc05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:26.284959 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-1d5dd376-9393-4bfc-8a2a-bc349a24fc05 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 06 14:44:26.284959 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:26.284959 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:26.284959 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:26.284959 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:26.284959 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:26.284959 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:26.284959 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:26.284959 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:26.284959 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:26.284959 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:26.284959 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:26.284959 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:26.284959 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:26.284959 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:26.284959 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:26.286246 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:26.286246 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:26.286246 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:26.286246 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:26.286246 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:26.286246 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:26.286246 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:26.286246 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:26.286246 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:26.286246 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:26.286246 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:26.286246 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:44:26.286246 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:26.286246 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:26.286246 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:26.286246 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:26.287632 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:26.287632 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:26.287632 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:26.287632 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:26.287632 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:26.287632 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:26.287632 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:26.287632 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:26.287632 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:26.287632 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:26.287632 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 06 14:44:26.287632 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:44:26.287632 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 140/287] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:44:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:44:26.293657 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5dd376-9393-4bfc-8a2a-bc349a24fc05 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:26.327455 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ef212-33ee-4ecc-b82b-3d1686198340 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:26.374069 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13bbbd-b48b-4086-a9e7-67a9bdf70275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:26.374728 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13bbbd-b48b-4086-a9e7-67a9bdf70275 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:26.375006 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13bbbd-b48b-4086-a9e7-67a9bdf70275 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:26.375542 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13bbbd-b48b-4086-a9e7-67a9bdf70275 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:26.425060 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5dd376-9393-4bfc-8a2a-bc349a24fc05 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:26.461924 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb50de81-691a-4bc8-8e38-d425d34bfd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:26.463346 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50de81-691a-4bc8-8e38-d425d34bfd22 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:26.463784 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50de81-691a-4bc8-8e38-d425d34bfd22 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:26.464404 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50de81-691a-4bc8-8e38-d425d34bfd22 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:26.465371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb50de81-691a-4bc8-8e38-d425d34bfd22 None admin] RBAC: Adding query filter params (name=reader) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:26.465826 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb50de81-691a-4bc8-8e38-d425d34bfd22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:26.467375 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb50de81-691a-4bc8-8e38-d425d34bfd22 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:26.477645 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 141/288] 10.222.0.23 () {56 vars in 1101 bytes} [Fri May 6 14:44:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:26.483964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb50de81-691a-4bc8-8e38-d425d34bfd22 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:26.505458 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc13bbbd-b48b-4086-a9e7-67a9bdf70275 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:26.507077 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc13bbbd-b48b-4086-a9e7-67a9bdf70275 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:26.602310 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb50de81-691a-4bc8-8e38-d425d34bfd22 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:26.654507 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc22214-211b-4641-af21-3a7e291b1195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:26.655387 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc22214-211b-4641-af21-3a7e291b1195 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:26.655666 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc22214-211b-4641-af21-3a7e291b1195 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:26.656191 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc22214-211b-4641-af21-3a7e291b1195 None admin] PATH_INFO: `/v3/users/1257f345ccf84d32980f775c05c3c3d2` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:26.658273 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc22214-211b-4641-af21-3a7e291b1195 None admin] RBAC: Authorizing `identity:get_user(user_id=1257f345ccf84d32980f775c05c3c3d2)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:26.659724 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc22214-211b-4641-af21-3a7e291b1195 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:26.661403 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 142/289] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:44:26 2022] GET /identity/v3/users/1257f345ccf84d32980f775c05c3c3d2 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:26.669690 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc22214-211b-4641-af21-3a7e291b1195 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:26.759832 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 148/290] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:44:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 06 14:44:26.764844 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13bbbd-b48b-4086-a9e7-67a9bdf70275 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:26.787145 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc22214-211b-4641-af21-3a7e291b1195 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:26.826535 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fa7ecb-4bee-49ed-8d72-cd05d48d3cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:26.827708 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa7ecb-4bee-49ed-8d72-cd05d48d3cf0 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:26.828185 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa7ecb-4bee-49ed-8d72-cd05d48d3cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:26.828645 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa7ecb-4bee-49ed-8d72-cd05d48d3cf0 None admin] PATH_INFO: `/v3/projects/643972018eac4175bcd3d9778028e93a` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:26.830305 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fa7ecb-4bee-49ed-8d72-cd05d48d3cf0 None admin] RBAC: Authorizing `identity:get_project(project_id=643972018eac4175bcd3d9778028e93a)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:26.831883 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fa7ecb-4bee-49ed-8d72-cd05d48d3cf0 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:26.834551 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 143/291] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:44:26 2022] GET /identity/v3/projects/643972018eac4175bcd3d9778028e93a => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:26.841105 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fa7ecb-4bee-49ed-8d72-cd05d48d3cf0 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:26.954845 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fa7ecb-4bee-49ed-8d72-cd05d48d3cf0 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:26.992823 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35077f7-1e77-41c4-baba-2a8ed2682141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:26.994138 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35077f7-1e77-41c4-baba-2a8ed2682141 None admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:26.994573 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35077f7-1e77-41c4-baba-2a8ed2682141 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:26.995373 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35077f7-1e77-41c4-baba-2a8ed2682141 None admin] PATH_INFO: `/v3/projects/643972018eac4175bcd3d9778028e93a/users/1257f345ccf84d32980f775c05c3c3d2/roles/e5c0610b69d640e595dccba09f44b81e` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:26.997724 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35077f7-1e77-41c4-baba-2a8ed2682141 None admin] RBAC: Authorizing `identity:create_grant(project_id=643972018eac4175bcd3d9778028e93a, user_id=1257f345ccf84d32980f775c05c3c3d2, role_id=e5c0610b69d640e595dccba09f44b81e)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:26.999427 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35077f7-1e77-41c4-baba-2a8ed2682141 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:27.015955 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 144/292] 10.222.0.23 () {58 vars in 1439 bytes} [Fri May 6 14:44:26 2022] PUT /identity/v3/projects/643972018eac4175bcd3d9778028e93a/users/1257f345ccf84d32980f775c05c3c3d2/roles/e5c0610b69d640e595dccba09f44b81e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 14:44:27.019903 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13bbbd-b48b-4086-a9e7-67a9bdf70275 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:27.091177 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdcfecd-2c38-44e4-9a43-ba815bbb9ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:27.092062 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdcfecd-2c38-44e4-9a43-ba815bbb9ec2 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:27.092346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdcfecd-2c38-44e4-9a43-ba815bbb9ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:27.092642 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdcfecd-2c38-44e4-9a43-ba815bbb9ec2 None admin] PATH_INFO: `/v3/projects/643972018eac4175bcd3d9778028e93a` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:27.097546 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdcfecd-2c38-44e4-9a43-ba815bbb9ec2 None admin] RBAC: Authorizing `identity:get_project(project_id=643972018eac4175bcd3d9778028e93a)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:27.099069 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdcfecd-2c38-44e4-9a43-ba815bbb9ec2 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:27.101197 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 149/293] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:44:26 2022] GET /identity/v3/projects/643972018eac4175bcd3d9778028e93a => generated 302 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:27.107256 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35077f7-1e77-41c4-baba-2a8ed2682141 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:27.261754 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35077f7-1e77-41c4-baba-2a8ed2682141 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:27.299657 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4220041-16bd-4302-ae24-a156fc5d5f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:27.300639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4220041-16bd-4302-ae24-a156fc5d5f68 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:27.301073 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4220041-16bd-4302-ae24-a156fc5d5f68 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:27.301556 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4220041-16bd-4302-ae24-a156fc5d5f68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:27.302535 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4220041-16bd-4302-ae24-a156fc5d5f68 None admin] RBAC: Adding query filter params (role.id=5f76e2e820b64d14b893c679427f4bed, scope.project.id=643972018eac4175bcd3d9778028e93a, user.id=8f144e48c3f246ec8954f36bc0624707) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:27.303019 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4220041-16bd-4302-ae24-a156fc5d5f68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:27.304684 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4220041-16bd-4302-ae24-a156fc5d5f68 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:27.311275 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 145/294] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:44:27 2022] GET /identity/v3/role_assignments?user.id=8f144e48c3f246ec8954f36bc0624707&scope.project.id=643972018eac4175bcd3d9778028e93a&role.id=5f76e2e820b64d14b893c679427f4bed => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:28.072872 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25262182-a3e0-41d8-bc0d-bac63f55dfec None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:28.073388 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25262182-a3e0-41d8-bc0d-bac63f55dfec None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:28.073493 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25262182-a3e0-41d8-bc0d-bac63f55dfec None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:28.075027 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 150/295] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:28.080446 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4703a8-a0d7-492a-9acd-71771262322d None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:28.080859 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4703a8-a0d7-492a-9acd-71771262322d None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:28.081045 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4703a8-a0d7-492a-9acd-71771262322d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:28.098388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-2b4703a8-a0d7-492a-9acd-71771262322d None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:28.099587 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-2b4703a8-a0d7-492a-9acd-71771262322d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:28.170049 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 146/296] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:44:28.191806 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25262182-a3e0-41d8-bc0d-bac63f55dfec None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:28.306155 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25262182-a3e0-41d8-bc0d-bac63f55dfec None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:28.357917 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a8c95f-2cec-48ff-a327-59e1c8367649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:28.358644 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a8c95f-2cec-48ff-a327-59e1c8367649 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:28.358929 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a8c95f-2cec-48ff-a327-59e1c8367649 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:28.359207 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a8c95f-2cec-48ff-a327-59e1c8367649 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:28.370380 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a8c95f-2cec-48ff-a327-59e1c8367649 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:28.378124 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-55a8c95f-2cec-48ff-a327-59e1c8367649 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 06 14:44:28.378124 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:28.378124 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:28.378124 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:28.378124 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:28.378124 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:28.378124 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:28.378124 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:28.378124 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:28.378124 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:28.378124 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:28.378124 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:28.378124 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:28.378124 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:28.378124 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:28.378124 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:28.378995 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:28.378995 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:28.378995 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:28.378995 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:28.378995 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:28.378995 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:28.378995 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:28.378995 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:28.378995 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:28.378995 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:28.378995 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:28.378995 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:44:28.378995 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:28.378995 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:28.378995 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:28.378995 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:28.379774 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:28.379774 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:28.379774 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:28.379774 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:28.379774 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:28.379774 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:28.379774 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:28.379774 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:28.379774 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:28.379774 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:28.379774 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 06 14:44:28.379774 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:44:28.379774 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 151/297] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:44:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:28.386441 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4703a8-a0d7-492a-9acd-71771262322d None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:28.486324 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4703a8-a0d7-492a-9acd-71771262322d None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:28.519159 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3dbc7b-a1dd-4dcb-aa2e-dce7d80fdcb8 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:28.519464 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3dbc7b-a1dd-4dcb-aa2e-dce7d80fdcb8 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:28.519719 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3dbc7b-a1dd-4dcb-aa2e-dce7d80fdcb8 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:28.521877 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 152/298] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:44:28.529547 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce19e861-87e5-4079-b1f6-064c94061fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:28.530232 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce19e861-87e5-4079-b1f6-064c94061fca None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:28.530519 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce19e861-87e5-4079-b1f6-064c94061fca None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:28.531058 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce19e861-87e5-4079-b1f6-064c94061fca None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:28.531816 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce19e861-87e5-4079-b1f6-064c94061fca None admin] RBAC: Adding query filter params (name=member) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:28.532126 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce19e861-87e5-4079-b1f6-064c94061fca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:28.532364 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39417a0d-5d2e-423f-8389-424bef9a523d None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:28.532875 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39417a0d-5d2e-423f-8389-424bef9a523d None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:28.533343 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39417a0d-5d2e-423f-8389-424bef9a523d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:28.533588 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce19e861-87e5-4079-b1f6-064c94061fca None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:28.548513 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 147/299] 10.222.0.23 () {56 vars in 1101 bytes} [Fri May 6 14:44:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:28.554671 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce19e861-87e5-4079-b1f6-064c94061fca None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:28.557853 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-39417a0d-5d2e-423f-8389-424bef9a523d None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:28.559075 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-39417a0d-5d2e-423f-8389-424bef9a523d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:28.640706 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 153/300] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 06 14:44:28.660732 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39417a0d-5d2e-423f-8389-424bef9a523d None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:28.697523 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce19e861-87e5-4079-b1f6-064c94061fca None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:28.732527 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce52d84-4492-418d-9074-98229eed7db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:28.733433 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce52d84-4492-418d-9074-98229eed7db4 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:28.733773 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce52d84-4492-418d-9074-98229eed7db4 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:28.733989 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce52d84-4492-418d-9074-98229eed7db4 None admin] PATH_INFO: `/v3/users/c93e3f6d8b514749bb8584fbf5282280` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:28.735273 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce52d84-4492-418d-9074-98229eed7db4 None admin] RBAC: Authorizing `identity:get_user(user_id=c93e3f6d8b514749bb8584fbf5282280)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:28.736658 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce52d84-4492-418d-9074-98229eed7db4 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:28.738289 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 148/301] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:44:28 2022] GET /identity/v3/users/c93e3f6d8b514749bb8584fbf5282280 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:28.744632 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce52d84-4492-418d-9074-98229eed7db4 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:28.782398 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39417a0d-5d2e-423f-8389-424bef9a523d None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:28.815642 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c59009f-517c-41fb-a144-a45e1d5cf7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:28.816446 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59009f-517c-41fb-a144-a45e1d5cf7b4 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:28.816759 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59009f-517c-41fb-a144-a45e1d5cf7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:28.817038 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59009f-517c-41fb-a144-a45e1d5cf7b4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:28.827628 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c59009f-517c-41fb-a144-a45e1d5cf7b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:28.835245 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-2c59009f-517c-41fb-a144-a45e1d5cf7b4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 06 14:44:28.835245 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:28.835245 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:28.835245 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:28.835245 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:28.835245 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:28.835245 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:28.835245 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:28.835245 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:28.835245 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:28.835245 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:28.835245 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:28.835245 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:28.835245 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:28.835245 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:28.835245 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:28.836818 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:28.836818 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:28.836818 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:28.836818 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:28.836818 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:28.836818 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:28.836818 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:28.836818 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:28.836818 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:28.836818 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:28.836818 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:28.836818 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:44:28.836818 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:28.836818 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:28.836818 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:28.836818 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:28.837669 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:28.837669 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:28.837669 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:28.837669 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:28.837669 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:28.837669 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:28.837669 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:28.837669 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:28.837669 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:28.837669 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:28.837669 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 06 14:44:28.837669 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:44:28.837669 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 154/302] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:44:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:44:28.838848 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae8b731-f3c6-4392-996a-748caab7edf3 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:28.839093 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae8b731-f3c6-4392-996a-748caab7edf3 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:28.839362 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae8b731-f3c6-4392-996a-748caab7edf3 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:28.844242 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 155/303] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:28.845171 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae8b731-f3c6-4392-996a-748caab7edf3 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:28.855832 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce52d84-4492-418d-9074-98229eed7db4 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:28.892017 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb120520-1d99-402d-a8a0-bd541b87e497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:28.892850 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb120520-1d99-402d-a8a0-bd541b87e497 None admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:28.893133 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb120520-1d99-402d-a8a0-bd541b87e497 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:28.893408 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb120520-1d99-402d-a8a0-bd541b87e497 None admin] PATH_INFO: `/v3/system/users/c93e3f6d8b514749bb8584fbf5282280/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:28.894880 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb120520-1d99-402d-a8a0-bd541b87e497 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c93e3f6d8b514749bb8584fbf5282280, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:28.896252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb120520-1d99-402d-a8a0-bd541b87e497 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:28.906835 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 149/304] 10.222.0.23 () {58 vars in 1334 bytes} [Fri May 6 14:44:28 2022] PUT /identity/v3/system/users/c93e3f6d8b514749bb8584fbf5282280/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:44:28.908491 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e8ba5-af9d-43e8-a03f-95e658d773e1 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:28.908874 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e8ba5-af9d-43e8-a03f-95e658d773e1 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:28.909226 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e8ba5-af9d-43e8-a03f-95e658d773e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:28.938349 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-905e8ba5-af9d-43e8-a03f-95e658d773e1 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:28.939765 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-905e8ba5-af9d-43e8-a03f-95e658d773e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:28.983529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae8b731-f3c6-4392-996a-748caab7edf3 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:29.028644 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 150/305] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:44:29.034050 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64034280-e22c-4797-8a51-5c3e2ebd6f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:29.034830 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64034280-e22c-4797-8a51-5c3e2ebd6f8d None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:29.035114 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64034280-e22c-4797-8a51-5c3e2ebd6f8d None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:29.035468 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64034280-e22c-4797-8a51-5c3e2ebd6f8d None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:29.036237 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64034280-e22c-4797-8a51-5c3e2ebd6f8d None admin] RBAC: Adding query filter params (name=reader) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:29.036582 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64034280-e22c-4797-8a51-5c3e2ebd6f8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:29.038072 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64034280-e22c-4797-8a51-5c3e2ebd6f8d None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:29.049097 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 156/306] 10.222.0.23 () {56 vars in 1101 bytes} [Fri May 6 14:44:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:29.049884 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e8ba5-af9d-43e8-a03f-95e658d773e1 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:29.054630 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64034280-e22c-4797-8a51-5c3e2ebd6f8d None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:29.160567 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64034280-e22c-4797-8a51-5c3e2ebd6f8d None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:29.162160 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e8ba5-af9d-43e8-a03f-95e658d773e1 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:29.195489 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d080aeb5-22e7-49c8-8681-cefc4b79bccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:29.196406 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d080aeb5-22e7-49c8-8681-cefc4b79bccf None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:29.196716 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d080aeb5-22e7-49c8-8681-cefc4b79bccf None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:29.196992 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d080aeb5-22e7-49c8-8681-cefc4b79bccf None admin] PATH_INFO: `/v3/users/6a5dae3b7c87419f9709d59afd6b0b7d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:29.199803 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7566b5f-3a07-46bf-9df4-bf683f4cb154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:29.200844 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7566b5f-3a07-46bf-9df4-bf683f4cb154 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:29.201324 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7566b5f-3a07-46bf-9df4-bf683f4cb154 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:29.201793 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7566b5f-3a07-46bf-9df4-bf683f4cb154 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:29.203185 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7566b5f-3a07-46bf-9df4-bf683f4cb154 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:29.204854 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7566b5f-3a07-46bf-9df4-bf683f4cb154 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:29.212095 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 151/307] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:44:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:29.212773 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d080aeb5-22e7-49c8-8681-cefc4b79bccf None admin] RBAC: Authorizing `identity:get_user(user_id=6a5dae3b7c87419f9709d59afd6b0b7d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:29.213793 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7566b5f-3a07-46bf-9df4-bf683f4cb154 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:29.214239 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d080aeb5-22e7-49c8-8681-cefc4b79bccf None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:29.216317 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 157/308] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:44:29 2022] GET /identity/v3/users/6a5dae3b7c87419f9709d59afd6b0b7d => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:29.221854 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d080aeb5-22e7-49c8-8681-cefc4b79bccf None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:29.321388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7566b5f-3a07-46bf-9df4-bf683f4cb154 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:29.328490 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d080aeb5-22e7-49c8-8681-cefc4b79bccf None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:29.356876 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bed329a-1c02-4beb-af53-8afa01acb58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:29.357851 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed329a-1c02-4beb-af53-8afa01acb58d None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:29.358393 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed329a-1c02-4beb-af53-8afa01acb58d None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:29.358670 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed329a-1c02-4beb-af53-8afa01acb58d None admin] PATH_INFO: `/v3/groups` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:29.359347 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bed329a-1c02-4beb-af53-8afa01acb58d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:29.360805 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bed329a-1c02-4beb-af53-8afa01acb58d None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:29.362305 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd08a74-c8ac-4d54-b000-2507a9065f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:29.363021 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd08a74-c8ac-4d54-b000-2507a9065f1a None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:29.363375 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd08a74-c8ac-4d54-b000-2507a9065f1a None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:29.363737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd08a74-c8ac-4d54-b000-2507a9065f1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:29.364647 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd08a74-c8ac-4d54-b000-2507a9065f1a None admin] RBAC: Adding query filter params (role.id=e5c0610b69d640e595dccba09f44b81e, user.id=6a5dae3b7c87419f9709d59afd6b0b7d, scope.system=all) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:29.365034 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd08a74-c8ac-4d54-b000-2507a9065f1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:29.367240 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd08a74-c8ac-4d54-b000-2507a9065f1a None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:29.377058 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 158/309] 10.222.0.23 () {56 vars in 1308 bytes} [Fri May 6 14:44:29 2022] GET /identity/v3/role_assignments?user.id=6a5dae3b7c87419f9709d59afd6b0b7d&scope.system=all&role.id=e5c0610b69d640e595dccba09f44b81e => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:29.377600 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 152/310] 10.222.0.23 () {60 vars in 1134 bytes} [Fri May 6 14:44:29 2022] POST /identity/v3/groups => generated 229 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 14:44:29.701448 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6641ea-ad15-409f-8438-998da4966b8e None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:29.701829 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6641ea-ad15-409f-8438-998da4966b8e None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:29.702013 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6641ea-ad15-409f-8438-998da4966b8e None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:29.703589 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 159/311] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:29.709073 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66dc104-96be-4cb6-86e8-ffaba6ab2ab5 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:29.709368 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66dc104-96be-4cb6-86e8-ffaba6ab2ab5 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:29.709649 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66dc104-96be-4cb6-86e8-ffaba6ab2ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:29.727071 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-e66dc104-96be-4cb6-86e8-ffaba6ab2ab5 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:29.728259 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-e66dc104-96be-4cb6-86e8-ffaba6ab2ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:29.797179 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 153/312] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 06 14:44:29.820957 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6641ea-ad15-409f-8438-998da4966b8e None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:29.925640 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6641ea-ad15-409f-8438-998da4966b8e None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:29.961399 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77fd71f-9127-4fac-8a3e-fbee10b07557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:29.962102 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77fd71f-9127-4fac-8a3e-fbee10b07557 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:29.962382 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77fd71f-9127-4fac-8a3e-fbee10b07557 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:29.962653 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77fd71f-9127-4fac-8a3e-fbee10b07557 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:29.973117 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77fd71f-9127-4fac-8a3e-fbee10b07557 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:29.981624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-b77fd71f-9127-4fac-8a3e-fbee10b07557 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 06 14:44:29.981624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:29.981624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:29.981624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:29.981624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:29.981624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:29.981624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:29.981624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:29.981624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:29.981624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:29.981624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:29.981624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:29.981624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:29.981624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:29.981624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:29.981624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:29.982503 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:29.982503 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:29.982503 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:29.982503 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:29.982503 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:29.982503 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:29.982503 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:29.982503 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:29.982503 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:29.982503 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:29.982503 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:29.982503 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:44:29.982503 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:29.982503 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:29.982503 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:29.982503 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:29.983455 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:29.983455 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:29.983455 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:29.983455 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:29.983455 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:29.983455 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:29.983455 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:29.983455 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:29.983455 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:29.983455 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:29.983455 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 06 14:44:29.983455 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:44:29.983455 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 160/313] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:44:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:29.988463 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66dc104-96be-4cb6-86e8-ffaba6ab2ab5 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:30.092843 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66dc104-96be-4cb6-86e8-ffaba6ab2ab5 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:30.127720 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ad3304-7073-4051-b72f-3d102739340d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:30.128392 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad3304-7073-4051-b72f-3d102739340d None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:30.128704 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad3304-7073-4051-b72f-3d102739340d None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:30.128982 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad3304-7073-4051-b72f-3d102739340d None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:30.129734 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ad3304-7073-4051-b72f-3d102739340d None admin] RBAC: Adding query filter params (name=reader) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:30.130046 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ad3304-7073-4051-b72f-3d102739340d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:30.131430 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ad3304-7073-4051-b72f-3d102739340d None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:30.140283 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 154/314] 10.222.0.23 () {56 vars in 1101 bytes} [Fri May 6 14:44:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:30.145964 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77fd71f-9127-4fac-8a3e-fbee10b07557 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:30.250972 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77fd71f-9127-4fac-8a3e-fbee10b07557 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:30.292836 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436645ba-864a-4c52-b1cc-c61037e5fcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:30.293712 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436645ba-864a-4c52-b1cc-c61037e5fcd8 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:30.293994 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436645ba-864a-4c52-b1cc-c61037e5fcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:30.294272 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436645ba-864a-4c52-b1cc-c61037e5fcd8 None admin] PATH_INFO: `/v3/users/1257f345ccf84d32980f775c05c3c3d2` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:30.295472 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436645ba-864a-4c52-b1cc-c61037e5fcd8 None admin] RBAC: Authorizing `identity:get_user(user_id=1257f345ccf84d32980f775c05c3c3d2)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:30.296932 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436645ba-864a-4c52-b1cc-c61037e5fcd8 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:30.298861 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 161/315] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:44:30 2022] GET /identity/v3/users/1257f345ccf84d32980f775c05c3c3d2 => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:30.304562 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ad3304-7073-4051-b72f-3d102739340d None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:30.404186 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ad3304-7073-4051-b72f-3d102739340d None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:30.437223 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d6a2b-7785-4437-a080-50656ddcd02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:30.437879 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d6a2b-7785-4437-a080-50656ddcd02a None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:30.438163 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d6a2b-7785-4437-a080-50656ddcd02a None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:30.438443 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d6a2b-7785-4437-a080-50656ddcd02a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:30.507915 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1d6a2b-7785-4437-a080-50656ddcd02a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:30.509466 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1d6a2b-7785-4437-a080-50656ddcd02a None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:30.615186 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 155/316] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:44:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 06 14:44:30.621442 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436645ba-864a-4c52-b1cc-c61037e5fcd8 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:30.724830 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436645ba-864a-4c52-b1cc-c61037e5fcd8 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:30.759686 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add83a1-7245-42f9-95e5-5439e6724261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:30.760468 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add83a1-7245-42f9-95e5-5439e6724261 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:30.761061 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add83a1-7245-42f9-95e5-5439e6724261 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:30.761347 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add83a1-7245-42f9-95e5-5439e6724261 None admin] PATH_INFO: `/v3/projects/643972018eac4175bcd3d9778028e93a` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:30.762542 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4add83a1-7245-42f9-95e5-5439e6724261 None admin] RBAC: Authorizing `identity:get_project(project_id=643972018eac4175bcd3d9778028e93a)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:30.764054 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4add83a1-7245-42f9-95e5-5439e6724261 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:30.766182 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 162/317] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:44:30 2022] GET /identity/v3/projects/643972018eac4175bcd3d9778028e93a => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:30.771957 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d6a2b-7785-4437-a080-50656ddcd02a None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:30.874961 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d6a2b-7785-4437-a080-50656ddcd02a None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:30.909762 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58c556-a3e7-4cfd-b0d9-ec95dad2566f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:30.910417 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58c556-a3e7-4cfd-b0d9-ec95dad2566f None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:30.910701 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58c556-a3e7-4cfd-b0d9-ec95dad2566f None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:30.910980 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58c556-a3e7-4cfd-b0d9-ec95dad2566f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:30.911808 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d58c556-a3e7-4cfd-b0d9-ec95dad2566f None admin] RBAC: Adding query filter params (role.id=e5c0610b69d640e595dccba09f44b81e, scope.project.id=643972018eac4175bcd3d9778028e93a, user.id=1257f345ccf84d32980f775c05c3c3d2) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:30.912123 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d58c556-a3e7-4cfd-b0d9-ec95dad2566f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:30.914124 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d58c556-a3e7-4cfd-b0d9-ec95dad2566f None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:30.920289 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 156/318] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:44:30 2022] GET /identity/v3/role_assignments?user.id=1257f345ccf84d32980f775c05c3c3d2&scope.project.id=643972018eac4175bcd3d9778028e93a&role.id=e5c0610b69d640e595dccba09f44b81e => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:31.308477 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14191d5-a0f6-478e-b25f-4c694912137d None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:31.309054 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14191d5-a0f6-478e-b25f-4c694912137d None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:31.309054 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14191d5-a0f6-478e-b25f-4c694912137d None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:31.310415 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 163/319] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:31.315845 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce33a2-aa22-488c-a5de-7880fdd2f768 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:31.316225 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce33a2-aa22-488c-a5de-7880fdd2f768 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:31.316472 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce33a2-aa22-488c-a5de-7880fdd2f768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:31.333490 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-a8ce33a2-aa22-488c-a5de-7880fdd2f768 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:31.334681 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-a8ce33a2-aa22-488c-a5de-7880fdd2f768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:31.403318 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 157/320] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:44:31.427804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14191d5-a0f6-478e-b25f-4c694912137d None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:31.533765 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14191d5-a0f6-478e-b25f-4c694912137d None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:31.573147 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eb2d8d-8889-4d0a-863c-7e0c6bb2a8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:31.573966 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb2d8d-8889-4d0a-863c-7e0c6bb2a8f3 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:31.574251 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb2d8d-8889-4d0a-863c-7e0c6bb2a8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:31.574530 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb2d8d-8889-4d0a-863c-7e0c6bb2a8f3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:31.585103 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eb2d8d-8889-4d0a-863c-7e0c6bb2a8f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:31.593344 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-38eb2d8d-8889-4d0a-863c-7e0c6bb2a8f3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 06 14:44:31.593344 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:31.593344 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:31.593344 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:31.593344 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:31.593344 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:31.593344 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:31.593344 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:31.593344 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:31.593344 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:31.593344 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:31.593344 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:31.593344 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:31.593344 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:31.593344 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:31.593344 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:31.594205 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:31.594205 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:31.594205 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:31.594205 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:31.594205 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:31.594205 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:31.594205 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:31.594205 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:31.594205 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:31.594205 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:31.594205 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:31.594205 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:44:31.594205 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:31.594205 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:31.594205 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:31.594205 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:31.595009 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:31.595009 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:31.595009 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:31.595009 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:31.595009 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:31.595009 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:31.595009 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:31.595009 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:31.595009 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:31.595009 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:31.595009 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 06 14:44:31.595009 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:44:31.596234 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 164/321] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:44:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:31.601762 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce33a2-aa22-488c-a5de-7880fdd2f768 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:31.702817 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce33a2-aa22-488c-a5de-7880fdd2f768 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:31.729670 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d506ce2-3a08-43fe-b7dd-45cfee070476 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:31.730068 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d506ce2-3a08-43fe-b7dd-45cfee070476 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:31.730427 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d506ce2-3a08-43fe-b7dd-45cfee070476 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:31.731891 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 165/322] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:44:31.736399 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce9203a-3090-43a5-9725-3ce9cd7ef4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:31.736813 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c66f9-4a1e-4c6d-96ed-42a6c9ea0779 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:31.737108 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce9203a-3090-43a5-9725-3ce9cd7ef4e9 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:31.737213 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c66f9-4a1e-4c6d-96ed-42a6c9ea0779 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:31.737391 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c66f9-4a1e-4c6d-96ed-42a6c9ea0779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:31.737498 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce9203a-3090-43a5-9725-3ce9cd7ef4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:31.737666 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce9203a-3090-43a5-9725-3ce9cd7ef4e9 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:31.738415 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce9203a-3090-43a5-9725-3ce9cd7ef4e9 None admin] RBAC: Adding query filter params (name=member) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:31.738721 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce9203a-3090-43a5-9725-3ce9cd7ef4e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:31.740093 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce9203a-3090-43a5-9725-3ce9cd7ef4e9 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:31.748456 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 158/323] 10.222.0.23 () {56 vars in 1101 bytes} [Fri May 6 14:44:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:31.749749 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ddaf1-0b50-410b-a710-41934328679a None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:31.750035 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ddaf1-0b50-410b-a710-41934328679a None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:31.750315 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ddaf1-0b50-410b-a710-41934328679a None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:31.751528 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 159/324] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:31.756036 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ddaf1-0b50-410b-a710-41934328679a None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:31.756164 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-f59c66f9-4a1e-4c6d-96ed-42a6c9ea0779 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:31.757433 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f59c66f9-4a1e-4c6d-96ed-42a6c9ea0779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:31.822335 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 166/325] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 06 14:44:31.824121 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbad93f-185c-480b-8777-e5b5219079cf None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:31.824121 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbad93f-185c-480b-8777-e5b5219079cf None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:31.824283 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbad93f-185c-480b-8777-e5b5219079cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:31.840711 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-4cbad93f-185c-480b-8777-e5b5219079cf None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:31.841892 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-4cbad93f-185c-480b-8777-e5b5219079cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:31.853260 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ddaf1-0b50-410b-a710-41934328679a None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:31.886362 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989473e6-6098-4bf0-ba76-774937a91dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:31.887220 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989473e6-6098-4bf0-ba76-774937a91dfa None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:31.887498 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989473e6-6098-4bf0-ba76-774937a91dfa None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:31.887768 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989473e6-6098-4bf0-ba76-774937a91dfa None admin] PATH_INFO: `/v3/users/c93e3f6d8b514749bb8584fbf5282280` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:31.888964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989473e6-6098-4bf0-ba76-774937a91dfa None admin] RBAC: Authorizing `identity:get_user(user_id=c93e3f6d8b514749bb8584fbf5282280)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:31.890399 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989473e6-6098-4bf0-ba76-774937a91dfa None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:31.892006 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 160/326] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:44:31 2022] GET /identity/v3/users/c93e3f6d8b514749bb8584fbf5282280 => generated 293 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:31.892788 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989473e6-6098-4bf0-ba76-774937a91dfa None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:31.906770 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 167/327] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:44:31.907603 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbad93f-185c-480b-8777-e5b5219079cf None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:31.989067 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989473e6-6098-4bf0-ba76-774937a91dfa None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:32.000931 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbad93f-185c-480b-8777-e5b5219079cf None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:32.021042 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79b00f5-8fa0-492a-9f29-a97cf25e372c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:32.021832 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b00f5-8fa0-492a-9f29-a97cf25e372c None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:32.022116 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b00f5-8fa0-492a-9f29-a97cf25e372c None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:32.022389 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b00f5-8fa0-492a-9f29-a97cf25e372c None admin] PATH_INFO: `/v3/domains/default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:32.023598 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79b00f5-8fa0-492a-9f29-a97cf25e372c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:32.025071 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79b00f5-8fa0-492a-9f29-a97cf25e372c None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:32.026679 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 161/328] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:44:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:32.027439 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79b00f5-8fa0-492a-9f29-a97cf25e372c None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:32.035017 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4797e343-e942-45cb-8ce7-b6235940588f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:32.035671 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4797e343-e942-45cb-8ce7-b6235940588f None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:32.035954 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4797e343-e942-45cb-8ce7-b6235940588f None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:32.036231 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4797e343-e942-45cb-8ce7-b6235940588f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:32.037090 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4797e343-e942-45cb-8ce7-b6235940588f None admin] RBAC: Adding query filter params (role.id=5f76e2e820b64d14b893c679427f4bed, user.id=c93e3f6d8b514749bb8584fbf5282280, scope.system=all) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:32.037399 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4797e343-e942-45cb-8ce7-b6235940588f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:32.039573 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4797e343-e942-45cb-8ce7-b6235940588f None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:32.050816 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 168/329] 10.222.0.23 () {56 vars in 1308 bytes} [Fri May 6 14:44:31 2022] GET /identity/v3/role_assignments?user.id=c93e3f6d8b514749bb8584fbf5282280&scope.system=all&role.id=5f76e2e820b64d14b893c679427f4bed => generated 510 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:32.051585 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4797e343-e942-45cb-8ce7-b6235940588f None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:32.133947 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79b00f5-8fa0-492a-9f29-a97cf25e372c None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:32.170291 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4797e343-e942-45cb-8ce7-b6235940588f None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:32.178470 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ab682-4fc3-4236-8f98-fc028b75eb78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:32.179368 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ab682-4fc3-4236-8f98-fc028b75eb78 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:32.179661 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ab682-4fc3-4236-8f98-fc028b75eb78 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:32.179936 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ab682-4fc3-4236-8f98-fc028b75eb78 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:32.191515 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105ab682-4fc3-4236-8f98-fc028b75eb78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:32.200132 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-105ab682-4fc3-4236-8f98-fc028b75eb78 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 06 14:44:32.200132 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:32.200132 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:32.200132 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:32.200132 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:32.200132 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:32.200132 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:32.200132 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:32.200132 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:32.200132 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:32.200132 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:32.200132 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:32.200132 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:32.200132 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:32.200132 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:32.200132 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:32.201384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:32.201384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:32.201384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:32.201384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:32.201384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:32.201384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:32.201384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:32.201384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:32.201384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:32.201384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:32.201384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:32.201384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:44:32.201384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:32.201384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:32.201384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:32.201384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:32.202751 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:32.202751 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:32.202751 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:32.202751 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:32.202751 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:32.202751 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:32.202751 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:32.202751 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:32.202751 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:32.202751 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:32.202751 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 06 14:44:32.202751 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:44:32.202751 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 162/330] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:44:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:32.207740 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ab682-4fc3-4236-8f98-fc028b75eb78 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:32.208880 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9336622a-a9ce-4306-bf37-eadcd42a366d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:32.209556 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336622a-a9ce-4306-bf37-eadcd42a366d None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:32.209839 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336622a-a9ce-4306-bf37-eadcd42a366d None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:32.210116 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336622a-a9ce-4306-bf37-eadcd42a366d None admin] PATH_INFO: `/v3/groups` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:32.210897 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9336622a-a9ce-4306-bf37-eadcd42a366d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:32.212903 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9336622a-a9ce-4306-bf37-eadcd42a366d None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:32.229289 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 169/331] 10.222.0.23 () {60 vars in 1134 bytes} [Fri May 6 14:44:32 2022] POST /identity/v3/groups => generated 226 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:44:32.336850 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ab682-4fc3-4236-8f98-fc028b75eb78 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:32.371328 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c846bf26-9e69-456c-a51f-6aedf92fbbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:32.371992 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846bf26-9e69-456c-a51f-6aedf92fbbb3 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:32.372275 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846bf26-9e69-456c-a51f-6aedf92fbbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:32.372553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846bf26-9e69-456c-a51f-6aedf92fbbb3 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:32.373337 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c846bf26-9e69-456c-a51f-6aedf92fbbb3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:32.373645 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c846bf26-9e69-456c-a51f-6aedf92fbbb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:32.375028 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c846bf26-9e69-456c-a51f-6aedf92fbbb3 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:32.384429 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 163/332] 10.222.0.23 () {56 vars in 1101 bytes} [Fri May 6 14:44:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:32.390298 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9336622a-a9ce-4306-bf37-eadcd42a366d None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:32.487922 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9336622a-a9ce-4306-bf37-eadcd42a366d None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:32.522695 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c164b9d-6ba5-48ed-9262-c602e2ed3479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:32.523609 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c164b9d-6ba5-48ed-9262-c602e2ed3479 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:32.523892 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c164b9d-6ba5-48ed-9262-c602e2ed3479 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:32.524167 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c164b9d-6ba5-48ed-9262-c602e2ed3479 None admin] PATH_INFO: `/v3/users/6a5dae3b7c87419f9709d59afd6b0b7d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:32.525399 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c164b9d-6ba5-48ed-9262-c602e2ed3479 None admin] RBAC: Authorizing `identity:get_user(user_id=6a5dae3b7c87419f9709d59afd6b0b7d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:32.526844 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c164b9d-6ba5-48ed-9262-c602e2ed3479 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:32.528644 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 170/333] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:44:32 2022] GET /identity/v3/users/6a5dae3b7c87419f9709d59afd6b0b7d => generated 293 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:32.534391 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c846bf26-9e69-456c-a51f-6aedf92fbbb3 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:32.641221 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c846bf26-9e69-456c-a51f-6aedf92fbbb3 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:32.681643 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a0a53b-d7a1-4422-b8a4-85c640910122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:32.682518 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0a53b-d7a1-4422-b8a4-85c640910122 None admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:32.682802 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0a53b-d7a1-4422-b8a4-85c640910122 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:32.683078 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0a53b-d7a1-4422-b8a4-85c640910122 None admin] PATH_INFO: `/v3/system/users/6a5dae3b7c87419f9709d59afd6b0b7d/roles/e5c0610b69d640e595dccba09f44b81e` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:32.685584 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a0a53b-d7a1-4422-b8a4-85c640910122 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6a5dae3b7c87419f9709d59afd6b0b7d, role_id=e5c0610b69d640e595dccba09f44b81e)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:32.686622 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a0a53b-d7a1-4422-b8a4-85c640910122 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:32.696273 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 164/334] 10.222.0.23 () {58 vars in 1334 bytes} [Fri May 6 14:44:32 2022] PUT /identity/v3/system/users/6a5dae3b7c87419f9709d59afd6b0b7d/roles/e5c0610b69d640e595dccba09f44b81e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:44:35.076322 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8dc8eb-1457-4913-86ac-6d34fa8a6c42 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:35.076919 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8dc8eb-1457-4913-86ac-6d34fa8a6c42 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:35.076919 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8dc8eb-1457-4913-86ac-6d34fa8a6c42 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:35.078533 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 171/335] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:35.084208 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8798d5d7-e206-4c59-8534-322d4a83ef0f None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:35.084680 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8798d5d7-e206-4c59-8534-322d4a83ef0f None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:35.084871 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8798d5d7-e206-4c59-8534-322d4a83ef0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:35.105811 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-8798d5d7-e206-4c59-8534-322d4a83ef0f None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:35.106998 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-8798d5d7-e206-4c59-8534-322d4a83ef0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:35.133476 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d48b49-5a13-4cbf-bfc9-f842444b954c None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:35.134607 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d48b49-5a13-4cbf-bfc9-f842444b954c None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:35.135120 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d48b49-5a13-4cbf-bfc9-f842444b954c None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:35.137451 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 172/336] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:35.144293 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c05dd-4251-4c8c-9160-5096a01f54b4 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:35.145011 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c05dd-4251-4c8c-9160-5096a01f54b4 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:35.145598 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c05dd-4251-4c8c-9160-5096a01f54b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:35.172811 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-606c05dd-4251-4c8c-9160-5096a01f54b4 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:35.174689 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-606c05dd-4251-4c8c-9160-5096a01f54b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:35.249577 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 173/337] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 06 14:44:35.256119 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 165/338] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:44:35.274964 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606c05dd-4251-4c8c-9160-5096a01f54b4 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:35.295250 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8798d5d7-e206-4c59-8534-322d4a83ef0f None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:35.387180 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606c05dd-4251-4c8c-9160-5096a01f54b4 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:35.426515 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ed67ad-84bf-4984-9bd4-acb7d46894f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:35.427569 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed67ad-84bf-4984-9bd4-acb7d46894f7 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:35.428025 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed67ad-84bf-4984-9bd4-acb7d46894f7 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:35.428453 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed67ad-84bf-4984-9bd4-acb7d46894f7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:35.439258 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ed67ad-84bf-4984-9bd4-acb7d46894f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:35.447325 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-90ed67ad-84bf-4984-9bd4-acb7d46894f7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 06 14:44:35.447325 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:35.447325 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:35.447325 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:35.447325 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:35.447325 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:35.447325 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:35.447325 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:35.447325 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:35.447325 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:35.447325 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:35.447325 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:35.447325 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:35.447325 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:35.447325 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:35.447325 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:35.448224 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:35.448224 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:35.448224 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:35.448224 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:35.448224 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:35.448224 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:35.448224 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:35.448224 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:35.448224 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:35.448224 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:35.448224 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:35.448224 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:44:35.448224 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:35.448224 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:35.448224 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:35.448224 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:35.449059 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:35.449059 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:35.449059 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:35.449059 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:35.449059 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:35.449059 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:35.449059 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:35.449059 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:35.449059 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:35.449059 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:35.449059 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 06 14:44:35.449059 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:44:35.451772 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 174/339] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:44:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:35.457325 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926544db-5abb-4c7c-9300-ff8bac87f1a6 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:35.457871 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926544db-5abb-4c7c-9300-ff8bac87f1a6 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:35.458333 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926544db-5abb-4c7c-9300-ff8bac87f1a6 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:35.459853 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 175/340] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:35.462133 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926544db-5abb-4c7c-9300-ff8bac87f1a6 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:35.484058 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8798d5d7-e206-4c59-8534-322d4a83ef0f None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:35.531113 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b0a626-572a-421e-8485-41ead3a8906b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:35.531918 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0a626-572a-421e-8485-41ead3a8906b None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:35.532412 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0a626-572a-421e-8485-41ead3a8906b None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:35.532898 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0a626-572a-421e-8485-41ead3a8906b None admin] PATH_INFO: `/v3/roles/member` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:35.561214 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b0a626-572a-421e-8485-41ead3a8906b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:35.569342 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-b3b0a626-572a-421e-8485-41ead3a8906b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 06 14:44:35.569342 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:35.569342 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:35.569342 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:35.569342 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:35.569342 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:35.569342 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:35.569342 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:35.569342 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:35.569342 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:35.569342 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:35.569342 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:35.569342 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:35.569342 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:35.569342 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:35.569342 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:35.570235 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:35.570235 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:35.570235 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:35.570235 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:35.570235 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:35.570235 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:35.570235 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:35.570235 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:35.570235 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:35.570235 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:35.570235 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:35.570235 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:44:35.570235 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:35.570235 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:35.570235 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:35.570235 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:35.571043 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:35.571043 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:35.571043 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:35.571043 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:35.571043 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:35.571043 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:35.571043 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:35.571043 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:35.571043 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:35.571043 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:35.571043 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 06 14:44:35.571043 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:44:35.577611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926544db-5abb-4c7c-9300-ff8bac87f1a6 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:35.578052 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 166/341] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:44:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:44:35.579483 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155af817-9202-4a48-98f2-084d3384b1cf None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:35.579971 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155af817-9202-4a48-98f2-084d3384b1cf None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:35.580420 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155af817-9202-4a48-98f2-084d3384b1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:35.613578 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-155af817-9202-4a48-98f2-084d3384b1cf None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:35.614786 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5efe645-2da8-40af-8e9e-2ee33cdec68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:35.615131 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-155af817-9202-4a48-98f2-084d3384b1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:35.615848 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5efe645-2da8-40af-8e9e-2ee33cdec68d None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:35.616279 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5efe645-2da8-40af-8e9e-2ee33cdec68d None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:35.616810 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5efe645-2da8-40af-8e9e-2ee33cdec68d None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:35.617703 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5efe645-2da8-40af-8e9e-2ee33cdec68d None admin] RBAC: Adding query filter params (name=member) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:35.618169 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5efe645-2da8-40af-8e9e-2ee33cdec68d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:35.619714 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5efe645-2da8-40af-8e9e-2ee33cdec68d None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:35.629432 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 176/342] 10.222.0.23 () {56 vars in 1101 bytes} [Fri May 6 14:44:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:35.630463 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5efe645-2da8-40af-8e9e-2ee33cdec68d None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:35.739598 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5efe645-2da8-40af-8e9e-2ee33cdec68d None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:35.763415 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 167/343] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:44:35.764348 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155af817-9202-4a48-98f2-084d3384b1cf None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:35.776278 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dea9c1-a48a-4b17-84f1-36a870c20cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:35.776983 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dea9c1-a48a-4b17-84f1-36a870c20cb0 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:35.777270 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dea9c1-a48a-4b17-84f1-36a870c20cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:35.777545 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dea9c1-a48a-4b17-84f1-36a870c20cb0 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:35.778288 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dea9c1-a48a-4b17-84f1-36a870c20cb0 None admin] RBAC: Adding query filter params (name=member) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:35.778595 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dea9c1-a48a-4b17-84f1-36a870c20cb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:35.779960 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dea9c1-a48a-4b17-84f1-36a870c20cb0 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:35.789121 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 177/344] 10.222.0.23 () {56 vars in 1101 bytes} [Fri May 6 14:44:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:35.794583 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dea9c1-a48a-4b17-84f1-36a870c20cb0 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:35.906361 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dea9c1-a48a-4b17-84f1-36a870c20cb0 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:35.943754 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c7d8a6-9a2f-4218-8e36-86a6a30e69e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:35.945291 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7d8a6-9a2f-4218-8e36-86a6a30e69e6 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:35.945722 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7d8a6-9a2f-4218-8e36-86a6a30e69e6 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:35.946139 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7d8a6-9a2f-4218-8e36-86a6a30e69e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:35.988260 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155af817-9202-4a48-98f2-084d3384b1cf None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:36.019244 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c7d8a6-9a2f-4218-8e36-86a6a30e69e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:36.021086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c7d8a6-9a2f-4218-8e36-86a6a30e69e6 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:36.057256 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ec903a-c3e7-41d4-b504-66d21f69228e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:36.058156 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec903a-c3e7-41d4-b504-66d21f69228e None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:36.058684 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec903a-c3e7-41d4-b504-66d21f69228e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:36.059105 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec903a-c3e7-41d4-b504-66d21f69228e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:36.138507 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ec903a-c3e7-41d4-b504-66d21f69228e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:36.140425 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 178/345] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:44:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 06 14:44:36.144964 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c7d8a6-9a2f-4218-8e36-86a6a30e69e6 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:36.145401 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ec903a-c3e7-41d4-b504-66d21f69228e None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:36.243564 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c7d8a6-9a2f-4218-8e36-86a6a30e69e6 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:36.262221 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 168/346] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:44:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 06 14:44:36.263997 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579e6db-9cdb-405a-b994-0c4e8fe3a11c None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:36.264455 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579e6db-9cdb-405a-b994-0c4e8fe3a11c None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:36.264898 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579e6db-9cdb-405a-b994-0c4e8fe3a11c None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:36.266302 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 169/347] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:36.267120 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1579e6db-9cdb-405a-b994-0c4e8fe3a11c None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:36.282925 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d28ec0a-fb11-4800-b4bd-26b485e020f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:36.283673 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28ec0a-fb11-4800-b4bd-26b485e020f5 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:36.283959 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28ec0a-fb11-4800-b4bd-26b485e020f5 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:36.284237 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28ec0a-fb11-4800-b4bd-26b485e020f5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:36.295620 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d28ec0a-fb11-4800-b4bd-26b485e020f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:36.304651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-1d28ec0a-fb11-4800-b4bd-26b485e020f5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 06 14:44:36.304651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:36.304651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:36.304651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:36.304651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:36.304651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:36.304651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:36.304651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:36.304651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:36.304651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:36.304651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:36.304651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:36.304651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:36.304651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:36.304651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:36.304651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:36.305804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:36.305804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:36.305804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:36.305804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:36.305804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:36.305804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:36.305804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:36.305804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:36.305804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:36.305804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:36.305804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:36.305804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:44:36.305804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:36.305804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:36.305804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:36.305804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:36.306671 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:36.306671 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:36.306671 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:36.306671 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:36.306671 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:36.306671 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:36.306671 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:36.306671 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:36.306671 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:36.306671 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:36.306671 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 06 14:44:36.306671 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:44:36.308595 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 179/348] 10.222.0.23 () {56 vars in 1114 bytes} [Fri May 6 14:44:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:36.309533 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d28ec0a-fb11-4800-b4bd-26b485e020f5 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:36.402195 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1579e6db-9cdb-405a-b994-0c4e8fe3a11c None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:36.437523 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d28ec0a-fb11-4800-b4bd-26b485e020f5 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:36.445185 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4210ee33-26bf-4abe-b557-a09800eaa61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:36.446310 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210ee33-26bf-4abe-b557-a09800eaa61c None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:36.446747 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210ee33-26bf-4abe-b557-a09800eaa61c None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:36.447178 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210ee33-26bf-4abe-b557-a09800eaa61c None admin] PATH_INFO: `/v3/projects/92337f7cabcc48ff8ca007298d1c3d78` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:36.449277 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4210ee33-26bf-4abe-b557-a09800eaa61c None admin] RBAC: Authorizing `identity:get_project(project_id=92337f7cabcc48ff8ca007298d1c3d78)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:36.450863 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4210ee33-26bf-4abe-b557-a09800eaa61c None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:36.452822 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 170/349] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:44:36 2022] GET /identity/v3/projects/92337f7cabcc48ff8ca007298d1c3d78 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:36.454210 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bbf48-e5c5-47e4-9506-b1f89d4874a4 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:36.454663 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bbf48-e5c5-47e4-9506-b1f89d4874a4 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:36.455131 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bbf48-e5c5-47e4-9506-b1f89d4874a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:36.482264 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-789bbf48-e5c5-47e4-9506-b1f89d4874a4 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:36.483690 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-789bbf48-e5c5-47e4-9506-b1f89d4874a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:36.484316 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e10cbe9-b4e5-4b21-90c2-72038962c7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:36.485217 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10cbe9-b4e5-4b21-90c2-72038962c7af None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:36.485494 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10cbe9-b4e5-4b21-90c2-72038962c7af None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:36.485762 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10cbe9-b4e5-4b21-90c2-72038962c7af None admin] PATH_INFO: `/v3/projects/643972018eac4175bcd3d9778028e93a` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:36.487836 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e10cbe9-b4e5-4b21-90c2-72038962c7af None admin] RBAC: Authorizing `identity:get_project(project_id=643972018eac4175bcd3d9778028e93a)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:36.490177 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e10cbe9-b4e5-4b21-90c2-72038962c7af None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:36.492265 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 180/350] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:44:36 2022] GET /identity/v3/projects/643972018eac4175bcd3d9778028e93a => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:36.493122 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e10cbe9-b4e5-4b21-90c2-72038962c7af None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:36.582135 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 171/351] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:44:36.583300 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789bbf48-e5c5-47e4-9506-b1f89d4874a4 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:36.614784 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e10cbe9-b4e5-4b21-90c2-72038962c7af None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:36.653126 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43448448-aded-4100-9cf8-4bcd8d3d2fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:36.654060 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43448448-aded-4100-9cf8-4bcd8d3d2fe6 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:36.654547 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43448448-aded-4100-9cf8-4bcd8d3d2fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:36.655042 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43448448-aded-4100-9cf8-4bcd8d3d2fe6 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:36.656231 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43448448-aded-4100-9cf8-4bcd8d3d2fe6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:36.656785 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43448448-aded-4100-9cf8-4bcd8d3d2fe6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:36.658353 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43448448-aded-4100-9cf8-4bcd8d3d2fe6 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:36.673202 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 181/352] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:44:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:36.674124 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43448448-aded-4100-9cf8-4bcd8d3d2fe6 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:36.707535 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789bbf48-e5c5-47e4-9506-b1f89d4874a4 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:36.747766 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a57282a-aba7-4889-ba49-0d67d4513f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:36.748665 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57282a-aba7-4889-ba49-0d67d4513f13 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:36.748945 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57282a-aba7-4889-ba49-0d67d4513f13 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:36.749221 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57282a-aba7-4889-ba49-0d67d4513f13 None admin] PATH_INFO: `/v3/groups/94e1fd9f935349cd9d51c38cabcba69d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:36.758692 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a57282a-aba7-4889-ba49-0d67d4513f13 None admin] RBAC: Authorizing `identity:get_group(group_id=94e1fd9f935349cd9d51c38cabcba69d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:36.760147 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a57282a-aba7-4889-ba49-0d67d4513f13 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:36.762080 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 172/353] 10.222.0.23 () {56 vars in 1180 bytes} [Fri May 6 14:44:36 2022] GET /identity/v3/groups/94e1fd9f935349cd9d51c38cabcba69d => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:36.763167 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed65e48-604e-4163-9186-26bb49033d23 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:36.763620 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed65e48-604e-4163-9186-26bb49033d23 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:36.763898 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed65e48-604e-4163-9186-26bb49033d23 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:36.765145 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 173/354] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:36.765821 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed65e48-604e-4163-9186-26bb49033d23 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:36.804795 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43448448-aded-4100-9cf8-4bcd8d3d2fe6 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:36.856046 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7b038-305b-4e9c-980b-5bbb922ab270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:36.857234 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7b038-305b-4e9c-980b-5bbb922ab270 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:36.857234 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7b038-305b-4e9c-980b-5bbb922ab270 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:36.857446 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7b038-305b-4e9c-980b-5bbb922ab270 None admin] PATH_INFO: `/v3/groups/94e1fd9f935349cd9d51c38cabcba69d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:36.858665 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb7b038-305b-4e9c-980b-5bbb922ab270 None admin] RBAC: Authorizing `identity:get_group(group_id=94e1fd9f935349cd9d51c38cabcba69d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:36.860102 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb7b038-305b-4e9c-980b-5bbb922ab270 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:36.863108 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 182/355] 10.222.0.23 () {56 vars in 1180 bytes} [Fri May 6 14:44:36 2022] GET /identity/v3/groups/94e1fd9f935349cd9d51c38cabcba69d => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:36.864955 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7b038-305b-4e9c-980b-5bbb922ab270 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:36.913986 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed65e48-604e-4163-9186-26bb49033d23 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:36.954228 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc796a-8e9c-46ef-a872-bcad4446708b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:36.955328 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc796a-8e9c-46ef-a872-bcad4446708b None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:36.955608 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc796a-8e9c-46ef-a872-bcad4446708b None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:36.955879 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc796a-8e9c-46ef-a872-bcad4446708b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:36.969845 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fc796a-8e9c-46ef-a872-bcad4446708b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:36.978699 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-e8fc796a-8e9c-46ef-a872-bcad4446708b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 06 14:44:36.978699 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:36.978699 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:36.978699 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:36.978699 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:36.978699 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:36.978699 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:36.978699 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:36.978699 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:36.978699 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:36.978699 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:36.978699 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:36.978699 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:36.978699 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:36.978699 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:36.978699 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:36.979658 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:36.979658 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:36.979658 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:36.979658 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:36.979658 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:36.979658 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:36.979658 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:36.979658 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:36.979658 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:36.979658 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:36.979658 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:36.979658 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:44:36.979658 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:36.979658 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:36.979658 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:36.979658 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:36.980801 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:36.980801 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:36.980801 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:36.980801 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:36.980801 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:36.980801 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:36.980801 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:36.980801 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:36.980801 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:36.980801 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:36.980801 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 06 14:44:36.980801 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:44:36.980801 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 174/356] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:44:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:36.980801 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc796a-8e9c-46ef-a872-bcad4446708b None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:36.990526 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7b038-305b-4e9c-980b-5bbb922ab270 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:37.056057 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea9194e-fdcc-441f-bd33-5c867b71f036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:37.056743 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9194e-fdcc-441f-bd33-5c867b71f036 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:37.057191 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9194e-fdcc-441f-bd33-5c867b71f036 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:37.057470 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9194e-fdcc-441f-bd33-5c867b71f036 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:37.115403 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc796a-8e9c-46ef-a872-bcad4446708b None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:37.131734 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea9194e-fdcc-441f-bd33-5c867b71f036 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:37.133773 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea9194e-fdcc-441f-bd33-5c867b71f036 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:37.160768 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e2a2c-b2f6-44d9-a8d6-b1ef6636604a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:37.160768 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e2a2c-b2f6-44d9-a8d6-b1ef6636604a None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:37.160768 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e2a2c-b2f6-44d9-a8d6-b1ef6636604a None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:37.160768 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e2a2c-b2f6-44d9-a8d6-b1ef6636604a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:37.161489 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e2a2c-b2f6-44d9-a8d6-b1ef6636604a None admin] RBAC: Adding query filter params (group.id=94e1fd9f935349cd9d51c38cabcba69d, role.id=5f76e2e820b64d14b893c679427f4bed, scope.project.id=92337f7cabcc48ff8ca007298d1c3d78) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:37.161489 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e2a2c-b2f6-44d9-a8d6-b1ef6636604a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:37.161489 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e2a2c-b2f6-44d9-a8d6-b1ef6636604a None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:37.168335 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 175/357] 10.222.0.23 () {56 vars in 1376 bytes} [Fri May 6 14:44:36 2022] GET /identity/v3/role_assignments?group.id=94e1fd9f935349cd9d51c38cabcba69d&scope.project.id=92337f7cabcc48ff8ca007298d1c3d78&role.id=5f76e2e820b64d14b893c679427f4bed => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:37.169653 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489122bf-c907-4163-9f28-4f47c3c3f184 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:37.170177 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489122bf-c907-4163-9f28-4f47c3c3f184 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:37.170670 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489122bf-c907-4163-9f28-4f47c3c3f184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:37.189911 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-489122bf-c907-4163-9f28-4f47c3c3f184 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:37.191553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-489122bf-c907-4163-9f28-4f47c3c3f184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:37.257440 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 183/358] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:44:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 06 14:44:37.259034 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea9194e-fdcc-441f-bd33-5c867b71f036 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:37.284498 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 176/359] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:44:37.285630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489122bf-c907-4163-9f28-4f47c3c3f184 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:37.396495 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea9194e-fdcc-441f-bd33-5c867b71f036 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:37.439101 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489122bf-c907-4163-9f28-4f47c3c3f184 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:37.439290 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b353730-b86a-4277-a7f0-37637df190b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:37.439876 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b353730-b86a-4277-a7f0-37637df190b0 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:37.440162 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b353730-b86a-4277-a7f0-37637df190b0 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:37.440437 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b353730-b86a-4277-a7f0-37637df190b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:37.441457 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b353730-b86a-4277-a7f0-37637df190b0 None admin] RBAC: Adding query filter params (group.id=94e1fd9f935349cd9d51c38cabcba69d, role.id=5f76e2e820b64d14b893c679427f4bed, scope.project.id=643972018eac4175bcd3d9778028e93a) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:37.441991 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b353730-b86a-4277-a7f0-37637df190b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:37.443630 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b353730-b86a-4277-a7f0-37637df190b0 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:37.453995 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 184/360] 10.222.0.23 () {56 vars in 1376 bytes} [Fri May 6 14:44:37 2022] GET /identity/v3/role_assignments?group.id=94e1fd9f935349cd9d51c38cabcba69d&scope.project.id=643972018eac4175bcd3d9778028e93a&role.id=5f76e2e820b64d14b893c679427f4bed => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:37.454279 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ed9b6-d0b8-4f28-803d-fd68f1346364 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:37.456879 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ed9b6-d0b8-4f28-803d-fd68f1346364 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:37.456879 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ed9b6-d0b8-4f28-803d-fd68f1346364 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:37.456879 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 185/361] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:37.457854 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016ed9b6-d0b8-4f28-803d-fd68f1346364 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:37.485489 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce59c9de-f572-47cc-badf-6e33ea9f6338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:37.486178 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59c9de-f572-47cc-badf-6e33ea9f6338 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:37.486458 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59c9de-f572-47cc-badf-6e33ea9f6338 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:37.486732 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59c9de-f572-47cc-badf-6e33ea9f6338 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:37.487501 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce59c9de-f572-47cc-badf-6e33ea9f6338 None admin] RBAC: Adding query filter params (name=reader) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:37.487810 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce59c9de-f572-47cc-badf-6e33ea9f6338 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:37.489268 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce59c9de-f572-47cc-badf-6e33ea9f6338 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:37.504089 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 177/362] 10.222.0.23 () {56 vars in 1101 bytes} [Fri May 6 14:44:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:37.506443 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce59c9de-f572-47cc-badf-6e33ea9f6338 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:37.624563 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016ed9b6-d0b8-4f28-803d-fd68f1346364 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:37.642837 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce59c9de-f572-47cc-badf-6e33ea9f6338 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:37.677542 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20f6540-b039-4d5f-ae28-769f9b7c7622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:37.678765 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f6540-b039-4d5f-ae28-769f9b7c7622 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:37.679439 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94613c51-e0e5-4fed-ba5e-971f3aee1291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:37.680201 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94613c51-e0e5-4fed-ba5e-971f3aee1291 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:37.680547 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f6540-b039-4d5f-ae28-769f9b7c7622 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:37.680856 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f6540-b039-4d5f-ae28-769f9b7c7622 None admin] PATH_INFO: `/v3/projects/92337f7cabcc48ff8ca007298d1c3d78` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:37.681184 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94613c51-e0e5-4fed-ba5e-971f3aee1291 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:37.681727 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94613c51-e0e5-4fed-ba5e-971f3aee1291 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:37.682101 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20f6540-b039-4d5f-ae28-769f9b7c7622 None admin] RBAC: Authorizing `identity:get_project(project_id=92337f7cabcc48ff8ca007298d1c3d78)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:37.683537 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20f6540-b039-4d5f-ae28-769f9b7c7622 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:37.685436 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 186/363] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:44:37 2022] GET /identity/v3/projects/92337f7cabcc48ff8ca007298d1c3d78 => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:37.686429 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583f8ec-c67d-4dcd-b8aa-7159f76657bf None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:37.686714 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583f8ec-c67d-4dcd-b8aa-7159f76657bf None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:37.686990 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583f8ec-c67d-4dcd-b8aa-7159f76657bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:37.695913 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94613c51-e0e5-4fed-ba5e-971f3aee1291 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:37.704664 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-94613c51-e0e5-4fed-ba5e-971f3aee1291 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 06 14:44:37.704664 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:37.704664 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:37.704664 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:37.704664 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:37.704664 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:37.704664 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:37.704664 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:37.704664 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:37.704664 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:37.704664 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:37.704664 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:37.704664 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:37.704664 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:37.704664 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:37.704664 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:37.705588 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:37.705588 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:37.705588 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:37.705588 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:37.705588 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:37.705588 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:37.705588 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:37.705588 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:37.705588 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:37.705588 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:37.705588 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:37.705588 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:44:37.705588 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:37.705588 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:37.705588 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:37.705588 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:37.706374 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:37.706374 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:37.706374 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:37.706374 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:37.706374 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:37.706374 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:37.706374 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:37.706374 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:37.706374 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:37.706374 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:37.706374 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 06 14:44:37.706374 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:44:37.712625 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-b583f8ec-c67d-4dcd-b8aa-7159f76657bf None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:37.712625 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-b583f8ec-c67d-4dcd-b8aa-7159f76657bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:37.713025 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 178/364] 10.222.0.23 () {56 vars in 1114 bytes} [Fri May 6 14:44:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:37.714110 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94613c51-e0e5-4fed-ba5e-971f3aee1291 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:37.821738 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 187/365] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:44:37.822983 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b583f8ec-c67d-4dcd-b8aa-7159f76657bf None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:37.846304 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94613c51-e0e5-4fed-ba5e-971f3aee1291 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:37.892449 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2984f4ef-0774-4d4e-ac6a-a1dc413d4de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:37.895187 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984f4ef-0774-4d4e-ac6a-a1dc413d4de0 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:37.895518 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984f4ef-0774-4d4e-ac6a-a1dc413d4de0 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:37.895845 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984f4ef-0774-4d4e-ac6a-a1dc413d4de0 None admin] PATH_INFO: `/v3/users/6a5dae3b7c87419f9709d59afd6b0b7d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:37.897871 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2984f4ef-0774-4d4e-ac6a-a1dc413d4de0 None admin] RBAC: Authorizing `identity:get_user(user_id=6a5dae3b7c87419f9709d59afd6b0b7d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:37.899861 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2984f4ef-0774-4d4e-ac6a-a1dc413d4de0 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:37.903276 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 179/366] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:44:37 2022] GET /identity/v3/users/6a5dae3b7c87419f9709d59afd6b0b7d => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:37.904184 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2984f4ef-0774-4d4e-ac6a-a1dc413d4de0 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:37.935661 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b583f8ec-c67d-4dcd-b8aa-7159f76657bf None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:37.977859 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b40378-c830-475a-b14e-096f8c8dbb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:37.979108 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b40378-c830-475a-b14e-096f8c8dbb75 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:37.979401 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b40378-c830-475a-b14e-096f8c8dbb75 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:37.979682 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b40378-c830-475a-b14e-096f8c8dbb75 None admin] PATH_INFO: `/v3/groups/94e1fd9f935349cd9d51c38cabcba69d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:37.981176 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b40378-c830-475a-b14e-096f8c8dbb75 None admin] RBAC: Authorizing `identity:get_group(group_id=94e1fd9f935349cd9d51c38cabcba69d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:37.983452 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b40378-c830-475a-b14e-096f8c8dbb75 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:37.985491 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 188/367] 10.222.0.23 () {56 vars in 1180 bytes} [Fri May 6 14:44:37 2022] GET /identity/v3/groups/94e1fd9f935349cd9d51c38cabcba69d => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:37.986092 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b40378-c830-475a-b14e-096f8c8dbb75 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:38.026807 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2984f4ef-0774-4d4e-ac6a-a1dc413d4de0 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:38.062614 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a202113b-a292-4737-9434-9f06cfd29d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:38.063695 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202113b-a292-4737-9434-9f06cfd29d0d None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:38.064177 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202113b-a292-4737-9434-9f06cfd29d0d None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:38.064707 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202113b-a292-4737-9434-9f06cfd29d0d None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:38.067376 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a202113b-a292-4737-9434-9f06cfd29d0d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:38.067912 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a202113b-a292-4737-9434-9f06cfd29d0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:38.069574 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a202113b-a292-4737-9434-9f06cfd29d0d None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:38.083792 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 180/368] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:44:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:38.085319 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a202113b-a292-4737-9434-9f06cfd29d0d None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:38.111271 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b40378-c830-475a-b14e-096f8c8dbb75 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:38.162096 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b5062d-efaf-42f2-8c5e-d1ae23cc7054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:38.162915 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5062d-efaf-42f2-8c5e-d1ae23cc7054 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:38.163726 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5062d-efaf-42f2-8c5e-d1ae23cc7054 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:38.164189 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5062d-efaf-42f2-8c5e-d1ae23cc7054 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:38.175907 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b5062d-efaf-42f2-8c5e-d1ae23cc7054 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:38.185382 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-39b5062d-efaf-42f2-8c5e-d1ae23cc7054 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:44:38.185382 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:38.185382 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:38.185382 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:38.185382 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:38.185382 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:38.185382 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:38.185382 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:38.185382 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:38.185382 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:38.185382 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:38.185382 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:38.185382 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:38.185382 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:38.185382 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:38.185382 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:38.186336 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:38.186336 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:38.186336 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:38.186336 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:38.186336 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:38.186336 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:38.186336 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:38.186336 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:38.186336 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:38.186336 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:38.186336 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:38.186336 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:44:38.186336 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:38.186336 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:38.186336 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:38.186336 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:38.187730 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:38.187730 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:38.187730 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:38.187730 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:38.187730 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:38.187730 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:38.187730 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:38.187730 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:38.187730 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:38.187730 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:38.187730 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:44:38.187730 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:44:38.187730 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 189/369] 10.222.0.23 () {56 vars in 1096 bytes} [Fri May 6 14:44:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:38.187730 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b5062d-efaf-42f2-8c5e-d1ae23cc7054 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:38.196731 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a202113b-a292-4737-9434-9f06cfd29d0d None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:38.234981 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22de2e1-b107-4b09-aaa9-bc47b9641030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:38.235654 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22de2e1-b107-4b09-aaa9-bc47b9641030 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:38.235937 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22de2e1-b107-4b09-aaa9-bc47b9641030 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:38.236214 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22de2e1-b107-4b09-aaa9-bc47b9641030 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:38.238379 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22de2e1-b107-4b09-aaa9-bc47b9641030 None admin] RBAC: Adding query filter params (role.id=e5c0610b69d640e595dccba09f44b81e, user.id=6a5dae3b7c87419f9709d59afd6b0b7d, scope.system=all) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:38.238697 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22de2e1-b107-4b09-aaa9-bc47b9641030 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:38.240144 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22de2e1-b107-4b09-aaa9-bc47b9641030 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:38.259681 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 181/370] 10.222.0.23 () {56 vars in 1308 bytes} [Fri May 6 14:44:38 2022] GET /identity/v3/role_assignments?user.id=6a5dae3b7c87419f9709d59afd6b0b7d&scope.system=all&role.id=e5c0610b69d640e595dccba09f44b81e => generated 510 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:38.260863 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22de2e1-b107-4b09-aaa9-bc47b9641030 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:38.324560 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b5062d-efaf-42f2-8c5e-d1ae23cc7054 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:38.380541 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba34287e-feb9-48f2-991c-f1ceeb45e979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:38.381230 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34287e-feb9-48f2-991c-f1ceeb45e979 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:38.381517 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34287e-feb9-48f2-991c-f1ceeb45e979 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:38.381794 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34287e-feb9-48f2-991c-f1ceeb45e979 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:38.382871 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba34287e-feb9-48f2-991c-f1ceeb45e979 None admin] RBAC: Adding query filter params (group.id=94e1fd9f935349cd9d51c38cabcba69d, role.id=d328f8040f2247d083309d839d2ff947, scope.project.id=92337f7cabcc48ff8ca007298d1c3d78) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:38.383196 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba34287e-feb9-48f2-991c-f1ceeb45e979 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:38.384456 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba34287e-feb9-48f2-991c-f1ceeb45e979 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:38.390964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22de2e1-b107-4b09-aaa9-bc47b9641030 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:38.394999 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 190/371] 10.222.0.23 () {56 vars in 1376 bytes} [Fri May 6 14:44:38 2022] GET /identity/v3/role_assignments?group.id=94e1fd9f935349cd9d51c38cabcba69d&scope.project.id=92337f7cabcc48ff8ca007298d1c3d78&role.id=d328f8040f2247d083309d839d2ff947 => generated 260 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:38.395853 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba34287e-feb9-48f2-991c-f1ceeb45e979 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:38.447794 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b6e09-9d5e-4f15-86b0-a7cd87b98d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:38.448834 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b6e09-9d5e-4f15-86b0-a7cd87b98d05 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:38.449384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b6e09-9d5e-4f15-86b0-a7cd87b98d05 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:38.449852 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b6e09-9d5e-4f15-86b0-a7cd87b98d05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:38.553263 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba34287e-feb9-48f2-991c-f1ceeb45e979 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:38.603436 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1965b6d-cb5f-4c37-b8f8-0ccb64619608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:38.604120 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1965b6d-cb5f-4c37-b8f8-0ccb64619608 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:38.604406 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1965b6d-cb5f-4c37-b8f8-0ccb64619608 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:38.604714 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1965b6d-cb5f-4c37-b8f8-0ccb64619608 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:38.605472 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1965b6d-cb5f-4c37-b8f8-0ccb64619608 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:38.605841 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1965b6d-cb5f-4c37-b8f8-0ccb64619608 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:38.607150 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1965b6d-cb5f-4c37-b8f8-0ccb64619608 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:38.621772 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 191/372] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:44:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:38.628536 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1965b6d-cb5f-4c37-b8f8-0ccb64619608 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:38.649836 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7b6e09-9d5e-4f15-86b0-a7cd87b98d05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:38.651344 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7b6e09-9d5e-4f15-86b0-a7cd87b98d05 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:38.747537 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1965b6d-cb5f-4c37-b8f8-0ccb64619608 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:38.786660 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ce143-34fb-41a3-ab71-df67929818ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:38.787336 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ce143-34fb-41a3-ab71-df67929818ec None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:38.787623 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ce143-34fb-41a3-ab71-df67929818ec None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:38.787959 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ce143-34fb-41a3-ab71-df67929818ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:38.861753 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9ce143-34fb-41a3-ab71-df67929818ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:38.863240 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9ce143-34fb-41a3-ab71-df67929818ec None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:38.901972 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 182/373] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:44:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 641 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 06 14:44:38.908349 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b6e09-9d5e-4f15-86b0-a7cd87b98d05 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:38.969748 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 192/374] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:44:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 06 14:44:38.975317 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ce143-34fb-41a3-ab71-df67929818ec None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:39.076848 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ce143-34fb-41a3-ab71-df67929818ec None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:39.114878 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b986c9-563c-4678-af67-f902a87153a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:39.115745 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b986c9-563c-4678-af67-f902a87153a2 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:39.116032 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b986c9-563c-4678-af67-f902a87153a2 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:39.116429 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b986c9-563c-4678-af67-f902a87153a2 None admin] PATH_INFO: `/v3/projects/ccc1ae27e989468199e57e6588faed16` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:39.117369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b6e09-9d5e-4f15-86b0-a7cd87b98d05 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:39.117827 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b986c9-563c-4678-af67-f902a87153a2 None admin] RBAC: Authorizing `identity:get_project(project_id=ccc1ae27e989468199e57e6588faed16)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:39.119275 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b986c9-563c-4678-af67-f902a87153a2 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:39.121333 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 193/375] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:44:38 2022] GET /identity/v3/projects/ccc1ae27e989468199e57e6588faed16 => generated 344 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:39.126590 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b986c9-563c-4678-af67-f902a87153a2 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:39.177328 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3ad3d-9509-4df8-92ae-7baa805578fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:39.178218 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3ad3d-9509-4df8-92ae-7baa805578fc None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:39.178498 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3ad3d-9509-4df8-92ae-7baa805578fc None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:39.178844 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3ad3d-9509-4df8-92ae-7baa805578fc None admin] PATH_INFO: `/v3/projects/643972018eac4175bcd3d9778028e93a` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:39.180338 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b3ad3d-9509-4df8-92ae-7baa805578fc None admin] RBAC: Authorizing `identity:get_project(project_id=643972018eac4175bcd3d9778028e93a)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:39.181832 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b3ad3d-9509-4df8-92ae-7baa805578fc None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:39.183858 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 183/376] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:44:38 2022] GET /identity/v3/projects/643972018eac4175bcd3d9778028e93a => generated 302 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:39.189051 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3ad3d-9509-4df8-92ae-7baa805578fc None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:39.233393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b986c9-563c-4678-af67-f902a87153a2 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:39.267492 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6859d275-2ecc-49c6-ab9b-40538119592f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:39.268354 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859d275-2ecc-49c6-ab9b-40538119592f None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:39.268682 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859d275-2ecc-49c6-ab9b-40538119592f None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:39.268935 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859d275-2ecc-49c6-ab9b-40538119592f None admin] PATH_INFO: `/v3/groups/e4a546bfd06d45958a194d886a8b846e` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:39.276032 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6859d275-2ecc-49c6-ab9b-40538119592f None admin] RBAC: Authorizing `identity:get_group(group_id=e4a546bfd06d45958a194d886a8b846e)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:39.277666 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6859d275-2ecc-49c6-ab9b-40538119592f None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:39.279841 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 194/377] 10.222.0.23 () {56 vars in 1180 bytes} [Fri May 6 14:44:39 2022] GET /identity/v3/groups/e4a546bfd06d45958a194d886a8b846e => generated 229 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:39.286082 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6859d275-2ecc-49c6-ab9b-40538119592f None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:39.326989 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3ad3d-9509-4df8-92ae-7baa805578fc None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:39.367244 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa24de31-8895-4b8c-ada1-5bde40beb3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:39.368499 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24de31-8895-4b8c-ada1-5bde40beb3d4 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:39.369009 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24de31-8895-4b8c-ada1-5bde40beb3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:39.369884 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24de31-8895-4b8c-ada1-5bde40beb3d4 None admin] PATH_INFO: `/v3/groups/94e1fd9f935349cd9d51c38cabcba69d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:39.371522 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa24de31-8895-4b8c-ada1-5bde40beb3d4 None admin] RBAC: Authorizing `identity:get_group(group_id=94e1fd9f935349cd9d51c38cabcba69d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:39.373362 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa24de31-8895-4b8c-ada1-5bde40beb3d4 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:39.375373 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 184/378] 10.222.0.23 () {56 vars in 1180 bytes} [Fri May 6 14:44:39 2022] GET /identity/v3/groups/94e1fd9f935349cd9d51c38cabcba69d => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:39.382643 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa24de31-8895-4b8c-ada1-5bde40beb3d4 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:39.496135 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa24de31-8895-4b8c-ada1-5bde40beb3d4 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:39.514497 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6859d275-2ecc-49c6-ab9b-40538119592f None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:39.550622 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b678e1-84ef-48fb-9819-b3345f093ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:39.551634 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b678e1-84ef-48fb-9819-b3345f093ac7 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:39.552135 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b678e1-84ef-48fb-9819-b3345f093ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:39.552622 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b678e1-84ef-48fb-9819-b3345f093ac7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:39.553242 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a902a72-9a7d-4889-9f7a-ac6e0a053cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:39.553926 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a902a72-9a7d-4889-9f7a-ac6e0a053cc4 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:39.554223 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b678e1-84ef-48fb-9819-b3345f093ac7 None admin] RBAC: Adding query filter params (group.id=94e1fd9f935349cd9d51c38cabcba69d, role.id=d328f8040f2247d083309d839d2ff947, scope.project.id=643972018eac4175bcd3d9778028e93a) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:39.554733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b678e1-84ef-48fb-9819-b3345f093ac7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:39.554833 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a902a72-9a7d-4889-9f7a-ac6e0a053cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:39.555108 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a902a72-9a7d-4889-9f7a-ac6e0a053cc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:39.555935 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a902a72-9a7d-4889-9f7a-ac6e0a053cc4 None admin] RBAC: Adding query filter params (group.id=e4a546bfd06d45958a194d886a8b846e, role.id=6dcd35d1a83748759d6ef9f026077835, scope.project.id=ccc1ae27e989468199e57e6588faed16) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:39.556246 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a902a72-9a7d-4889-9f7a-ac6e0a053cc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:39.560623 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a902a72-9a7d-4889-9f7a-ac6e0a053cc4 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:39.561431 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b678e1-84ef-48fb-9819-b3345f093ac7 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:39.568069 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 185/379] 10.222.0.23 () {56 vars in 1376 bytes} [Fri May 6 14:44:39 2022] GET /identity/v3/role_assignments?group.id=94e1fd9f935349cd9d51c38cabcba69d&scope.project.id=643972018eac4175bcd3d9778028e93a&role.id=d328f8040f2247d083309d839d2ff947 => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:39.568968 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 195/380] 10.222.0.23 () {56 vars in 1376 bytes} [Fri May 6 14:44:39 2022] GET /identity/v3/role_assignments?group.id=e4a546bfd06d45958a194d886a8b846e&scope.project.id=ccc1ae27e989468199e57e6588faed16&role.id=6dcd35d1a83748759d6ef9f026077835 => generated 260 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:40.138300 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faeea53-09e2-430e-9eee-7a8234e6edb5 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:40.138673 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faeea53-09e2-430e-9eee-7a8234e6edb5 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:40.138850 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faeea53-09e2-430e-9eee-7a8234e6edb5 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:40.140378 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 186/381] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:40.145990 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d7cdc9-ff80-4c42-a920-775644108633 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:40.146278 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d7cdc9-ff80-4c42-a920-775644108633 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:40.146557 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d7cdc9-ff80-4c42-a920-775644108633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:40.170539 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-73d7cdc9-ff80-4c42-a920-775644108633 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:40.171735 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-73d7cdc9-ff80-4c42-a920-775644108633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:40.257396 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 196/382] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:44:40.294900 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faeea53-09e2-430e-9eee-7a8234e6edb5 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:40.394886 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278abfae-a75c-4025-a4ca-3dbff00cce17 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:40.395172 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278abfae-a75c-4025-a4ca-3dbff00cce17 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:40.395371 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278abfae-a75c-4025-a4ca-3dbff00cce17 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:40.396726 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 197/383] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:44:40.416756 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8da28b-0e95-4450-a5cf-3e231b286a31 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:40.417056 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8da28b-0e95-4450-a5cf-3e231b286a31 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:40.417310 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8da28b-0e95-4450-a5cf-3e231b286a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:40.433829 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faeea53-09e2-430e-9eee-7a8234e6edb5 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:40.440182 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-ca8da28b-0e95-4450-a5cf-3e231b286a31 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:40.441371 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-ca8da28b-0e95-4450-a5cf-3e231b286a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:40.497513 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfc6ec4-9d2a-4cf4-9cf0-26240c2391b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:40.498321 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc6ec4-9d2a-4cf4-9cf0-26240c2391b8 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:40.498601 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc6ec4-9d2a-4cf4-9cf0-26240c2391b8 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:40.498873 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc6ec4-9d2a-4cf4-9cf0-26240c2391b8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:40.518384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfc6ec4-9d2a-4cf4-9cf0-26240c2391b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:40.526574 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-8dfc6ec4-9d2a-4cf4-9cf0-26240c2391b8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 06 14:44:40.526574 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:40.526574 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:40.526574 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:40.526574 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:40.526574 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:40.526574 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:40.526574 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:40.526574 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:40.526574 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:40.526574 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:40.526574 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:40.526574 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:40.526574 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:40.526574 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:40.526574 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:40.527459 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:40.527459 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:40.527459 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:40.527459 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:40.527459 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:40.527459 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:40.527459 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:40.527459 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:40.527459 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:40.527459 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:40.527459 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:40.527459 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:44:40.527459 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:40.527459 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:40.527459 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:40.527459 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:40.528263 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:40.528263 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:40.528263 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:40.528263 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:40.528263 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:40.528263 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:40.528263 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:40.528263 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:40.528263 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:40.528263 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:40.528263 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 06 14:44:40.528263 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:44:40.529371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 187/384] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:44:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:40.531441 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 198/385] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 06 14:44:40.534219 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfc6ec4-9d2a-4cf4-9cf0-26240c2391b8 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:40.574404 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8da28b-0e95-4450-a5cf-3e231b286a31 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:40.641752 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfc6ec4-9d2a-4cf4-9cf0-26240c2391b8 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:40.680808 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9d29ad-8c70-4991-af8f-8f15efa7c58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:40.681407 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d29ad-8c70-4991-af8f-8f15efa7c58c None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:40.681688 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d29ad-8c70-4991-af8f-8f15efa7c58c None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:40.681960 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d29ad-8c70-4991-af8f-8f15efa7c58c None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:40.682711 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9d29ad-8c70-4991-af8f-8f15efa7c58c None admin] RBAC: Adding query filter params (name=member) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:40.683019 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9d29ad-8c70-4991-af8f-8f15efa7c58c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:40.684408 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9d29ad-8c70-4991-af8f-8f15efa7c58c None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:40.702746 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 188/386] 10.222.0.23 () {56 vars in 1101 bytes} [Fri May 6 14:44:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:40.704058 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9d29ad-8c70-4991-af8f-8f15efa7c58c None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:40.714496 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8da28b-0e95-4450-a5cf-3e231b286a31 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:40.752289 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8697be3c-cd6c-45a3-ab87-ff197c81669f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:40.753355 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8697be3c-cd6c-45a3-ab87-ff197c81669f None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:40.753774 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8697be3c-cd6c-45a3-ab87-ff197c81669f None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:40.754180 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8697be3c-cd6c-45a3-ab87-ff197c81669f None admin] PATH_INFO: `/v3/roles/member` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:40.766744 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8697be3c-cd6c-45a3-ab87-ff197c81669f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:40.775612 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-8697be3c-cd6c-45a3-ab87-ff197c81669f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 06 14:44:40.775612 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:40.775612 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:40.775612 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:40.775612 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:40.775612 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:40.775612 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:40.775612 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:40.775612 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:40.775612 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:40.775612 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:40.775612 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:40.775612 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:40.775612 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:40.775612 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:40.775612 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:40.776729 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:40.776729 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:40.776729 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:40.776729 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:40.776729 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:40.776729 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:40.776729 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:40.776729 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:40.776729 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:40.776729 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:40.776729 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:40.776729 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:44:40.776729 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:40.776729 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:40.776729 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:40.776729 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:40.777597 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:40.777597 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:40.777597 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:40.777597 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:40.777597 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:40.777597 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:40.777597 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:40.777597 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:40.777597 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:40.777597 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:40.777597 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 06 14:44:40.777597 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:44:40.779908 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 199/387] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:44:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:44:40.786664 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8697be3c-cd6c-45a3-ab87-ff197c81669f None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:40.815802 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9d29ad-8c70-4991-af8f-8f15efa7c58c None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:40.850091 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae99451c-ea01-4865-82a5-d83279397dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:40.851469 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae99451c-ea01-4865-82a5-d83279397dc4 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:40.851610 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae99451c-ea01-4865-82a5-d83279397dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:40.851843 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae99451c-ea01-4865-82a5-d83279397dc4 None admin] PATH_INFO: `/v3/groups/94e1fd9f935349cd9d51c38cabcba69d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:40.853070 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae99451c-ea01-4865-82a5-d83279397dc4 None admin] RBAC: Authorizing `identity:get_group(group_id=94e1fd9f935349cd9d51c38cabcba69d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:40.854529 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae99451c-ea01-4865-82a5-d83279397dc4 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:40.860660 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 189/388] 10.222.0.23 () {56 vars in 1180 bytes} [Fri May 6 14:44:40 2022] GET /identity/v3/groups/94e1fd9f935349cd9d51c38cabcba69d => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:40.861891 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae99451c-ea01-4865-82a5-d83279397dc4 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:40.897152 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8697be3c-cd6c-45a3-ab87-ff197c81669f None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:40.933953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab853f-4925-483f-9099-77025222aa6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:40.934869 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab853f-4925-483f-9099-77025222aa6c None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:40.935289 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab853f-4925-483f-9099-77025222aa6c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:40.935754 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab853f-4925-483f-9099-77025222aa6c None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:40.936648 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ab853f-4925-483f-9099-77025222aa6c None admin] RBAC: Adding query filter params (name=member) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:40.937138 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ab853f-4925-483f-9099-77025222aa6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:40.938688 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ab853f-4925-483f-9099-77025222aa6c None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:40.948258 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 200/389] 10.222.0.23 () {56 vars in 1101 bytes} [Fri May 6 14:44:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:40.954004 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab853f-4925-483f-9099-77025222aa6c None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:40.964480 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae99451c-ea01-4865-82a5-d83279397dc4 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:40.999182 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e821e0-4419-4c11-aa4d-837ed569a377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:40.999762 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e821e0-4419-4c11-aa4d-837ed569a377 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:41.000046 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e821e0-4419-4c11-aa4d-837ed569a377 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:41.000321 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e821e0-4419-4c11-aa4d-837ed569a377 None admin] PATH_INFO: `/v3/projects/643972018eac4175bcd3d9778028e93a` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:41.001598 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e821e0-4419-4c11-aa4d-837ed569a377 None admin] RBAC: Authorizing `identity:get_project(project_id=643972018eac4175bcd3d9778028e93a)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:41.003029 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e821e0-4419-4c11-aa4d-837ed569a377 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:41.008717 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 190/390] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:44:40 2022] GET /identity/v3/projects/643972018eac4175bcd3d9778028e93a => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:41.010690 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e821e0-4419-4c11-aa4d-837ed569a377 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:41.081711 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab853f-4925-483f-9099-77025222aa6c None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:41.112738 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e821e0-4419-4c11-aa4d-837ed569a377 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:41.135492 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a1ac3-f63b-48f3-a99c-6dc7da128a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:41.136719 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a1ac3-f63b-48f3-a99c-6dc7da128a19 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:41.137196 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a1ac3-f63b-48f3-a99c-6dc7da128a19 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:41.137658 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a1ac3-f63b-48f3-a99c-6dc7da128a19 None admin] PATH_INFO: `/v3/groups/94e1fd9f935349cd9d51c38cabcba69d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:41.139049 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4a1ac3-f63b-48f3-a99c-6dc7da128a19 None admin] RBAC: Authorizing `identity:get_group(group_id=94e1fd9f935349cd9d51c38cabcba69d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:41.140728 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4a1ac3-f63b-48f3-a99c-6dc7da128a19 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:41.142836 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 201/391] 10.222.0.23 () {56 vars in 1180 bytes} [Fri May 6 14:44:40 2022] GET /identity/v3/groups/94e1fd9f935349cd9d51c38cabcba69d => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:41.149127 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a1ac3-f63b-48f3-a99c-6dc7da128a19 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:41.149263 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e59332-ea20-4634-8f8e-11c7c63a735d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:41.152622 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e59332-ea20-4634-8f8e-11c7c63a735d None admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:41.152622 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e59332-ea20-4634-8f8e-11c7c63a735d None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:41.152622 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e59332-ea20-4634-8f8e-11c7c63a735d None admin] PATH_INFO: `/v3/projects/643972018eac4175bcd3d9778028e93a/groups/94e1fd9f935349cd9d51c38cabcba69d/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:41.152622 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e59332-ea20-4634-8f8e-11c7c63a735d None admin] RBAC: Authorizing `identity:create_grant(project_id=643972018eac4175bcd3d9778028e93a, group_id=94e1fd9f935349cd9d51c38cabcba69d, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:41.154054 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e59332-ea20-4634-8f8e-11c7c63a735d None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:41.187840 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 191/392] 10.222.0.23 () {58 vars in 1442 bytes} [Fri May 6 14:44:41 2022] PUT /identity/v3/projects/643972018eac4175bcd3d9778028e93a/groups/94e1fd9f935349cd9d51c38cabcba69d/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:44:41.266327 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a1ac3-f63b-48f3-a99c-6dc7da128a19 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:41.308299 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f7c798-4d81-4e88-a7b1-31aa12af84a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:41.309214 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7c798-4d81-4e88-a7b1-31aa12af84a2 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:41.309558 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7c798-4d81-4e88-a7b1-31aa12af84a2 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:41.309836 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7c798-4d81-4e88-a7b1-31aa12af84a2 None admin] PATH_INFO: `/v3/projects/92337f7cabcc48ff8ca007298d1c3d78` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:41.311132 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f7c798-4d81-4e88-a7b1-31aa12af84a2 None admin] RBAC: Authorizing `identity:get_project(project_id=92337f7cabcc48ff8ca007298d1c3d78)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:41.312646 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f7c798-4d81-4e88-a7b1-31aa12af84a2 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:41.314814 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 202/393] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:44:41 2022] GET /identity/v3/projects/92337f7cabcc48ff8ca007298d1c3d78 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:41.320984 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e59332-ea20-4634-8f8e-11c7c63a735d None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:41.484725 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e59332-ea20-4634-8f8e-11c7c63a735d None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:41.528069 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0413f5a4-847a-48c9-bb02-cf241b67570d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:41.529340 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413f5a4-847a-48c9-bb02-cf241b67570d None admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:41.529841 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413f5a4-847a-48c9-bb02-cf241b67570d None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:41.530328 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413f5a4-847a-48c9-bb02-cf241b67570d None admin] PATH_INFO: `/v3/projects/92337f7cabcc48ff8ca007298d1c3d78/groups/94e1fd9f935349cd9d51c38cabcba69d/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:41.532913 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0413f5a4-847a-48c9-bb02-cf241b67570d None admin] RBAC: Authorizing `identity:create_grant(project_id=92337f7cabcc48ff8ca007298d1c3d78, group_id=94e1fd9f935349cd9d51c38cabcba69d, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:41.534564 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0413f5a4-847a-48c9-bb02-cf241b67570d None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:41.551558 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 192/394] 10.222.0.23 () {58 vars in 1442 bytes} [Fri May 6 14:44:41 2022] PUT /identity/v3/projects/92337f7cabcc48ff8ca007298d1c3d78/groups/94e1fd9f935349cd9d51c38cabcba69d/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:44:41.777001 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b283749-2fd9-4f7f-ad19-a7f76040948c None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:41.777552 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b283749-2fd9-4f7f-ad19-a7f76040948c None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:41.777552 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b283749-2fd9-4f7f-ad19-a7f76040948c None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:41.779305 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 203/395] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:41.785310 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679970f-509e-4263-af2b-8b0c7b380af3 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:41.785909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679970f-509e-4263-af2b-8b0c7b380af3 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:41.786423 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679970f-509e-4263-af2b-8b0c7b380af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:41.811332 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-d679970f-509e-4263-af2b-8b0c7b380af3 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:41.812507 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d679970f-509e-4263-af2b-8b0c7b380af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:41.921903 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 193/396] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:44:41.957399 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b283749-2fd9-4f7f-ad19-a7f76040948c None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:42.105305 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b283749-2fd9-4f7f-ad19-a7f76040948c None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:42.146071 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3916a3a-7bf6-441c-9af3-69ad2b4c5b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:42.146889 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3916a3a-7bf6-441c-9af3-69ad2b4c5b5c None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:42.147173 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3916a3a-7bf6-441c-9af3-69ad2b4c5b5c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:42.147909 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3916a3a-7bf6-441c-9af3-69ad2b4c5b5c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:42.180158 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3916a3a-7bf6-441c-9af3-69ad2b4c5b5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:42.187685 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-d3916a3a-7bf6-441c-9af3-69ad2b4c5b5c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 06 14:44:42.187685 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:42.187685 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:42.187685 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:42.187685 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:42.187685 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:42.187685 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:42.187685 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:42.187685 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:42.187685 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:42.187685 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:42.187685 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:42.187685 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:42.187685 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:42.187685 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:42.187685 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:42.188563 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:42.188563 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:42.188563 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:42.188563 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:42.188563 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:42.188563 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:42.188563 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:42.188563 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:42.188563 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:42.188563 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:42.188563 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:42.188563 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:44:42.188563 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:42.188563 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:42.188563 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:42.188563 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:42.190101 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:42.190101 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:42.190101 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:42.190101 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:42.190101 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:42.190101 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:42.190101 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:42.190101 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:42.190101 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:42.190101 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:42.190101 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 06 14:44:42.190101 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:44:42.190101 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 204/397] 10.222.0.23 () {56 vars in 1114 bytes} [Fri May 6 14:44:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:42.197693 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679970f-509e-4263-af2b-8b0c7b380af3 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:42.360233 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679970f-509e-4263-af2b-8b0c7b380af3 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:42.405594 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f87dc4-da8c-4db6-91a3-fc0ce64cfab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:42.406527 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f87dc4-da8c-4db6-91a3-fc0ce64cfab8 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:42.406806 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f87dc4-da8c-4db6-91a3-fc0ce64cfab8 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:42.407078 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f87dc4-da8c-4db6-91a3-fc0ce64cfab8 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:42.407824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f87dc4-da8c-4db6-91a3-fc0ce64cfab8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:42.408127 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f87dc4-da8c-4db6-91a3-fc0ce64cfab8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:42.409520 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f87dc4-da8c-4db6-91a3-fc0ce64cfab8 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:42.424798 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 194/398] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:44:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:42.430447 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3916a3a-7bf6-441c-9af3-69ad2b4c5b5c None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:42.541522 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3916a3a-7bf6-441c-9af3-69ad2b4c5b5c None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:42.565996 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970cf5dd-4c11-4064-94dd-d1bfac6cd027 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:42.566631 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970cf5dd-4c11-4064-94dd-d1bfac6cd027 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:42.566631 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970cf5dd-4c11-4064-94dd-d1bfac6cd027 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:42.568083 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 195/399] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:44:42.576459 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062aedc-903f-4f46-83b1-b953d8e794b7 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:42.576459 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062aedc-903f-4f46-83b1-b953d8e794b7 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:42.576459 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062aedc-903f-4f46-83b1-b953d8e794b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:42.577534 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd4cde-072e-4f52-b5b4-12f9cab7c558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:42.580776 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd4cde-072e-4f52-b5b4-12f9cab7c558 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:42.580776 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd4cde-072e-4f52-b5b4-12f9cab7c558 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:42.580776 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd4cde-072e-4f52-b5b4-12f9cab7c558 None admin] PATH_INFO: `/v3/groups/94e1fd9f935349cd9d51c38cabcba69d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:42.582550 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd4cde-072e-4f52-b5b4-12f9cab7c558 None admin] RBAC: Authorizing `identity:get_group(group_id=94e1fd9f935349cd9d51c38cabcba69d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:42.584819 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd4cde-072e-4f52-b5b4-12f9cab7c558 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:42.588422 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 205/400] 10.222.0.23 () {56 vars in 1180 bytes} [Fri May 6 14:44:42 2022] GET /identity/v3/groups/94e1fd9f935349cd9d51c38cabcba69d => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:42.594502 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd4cde-072e-4f52-b5b4-12f9cab7c558 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:42.617798 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-e062aedc-903f-4f46-83b1-b953d8e794b7 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:42.619073 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-e062aedc-903f-4f46-83b1-b953d8e794b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:42.708132 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd4cde-072e-4f52-b5b4-12f9cab7c558 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:42.753363 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dc4114-e1fb-4b8a-87f4-ef38e1d5a2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:42.754234 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dc4114-e1fb-4b8a-87f4-ef38e1d5a2d4 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:42.754512 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dc4114-e1fb-4b8a-87f4-ef38e1d5a2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:42.754788 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dc4114-e1fb-4b8a-87f4-ef38e1d5a2d4 None admin] PATH_INFO: `/v3/projects/92337f7cabcc48ff8ca007298d1c3d78` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:42.755972 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dc4114-e1fb-4b8a-87f4-ef38e1d5a2d4 None admin] RBAC: Authorizing `identity:get_project(project_id=92337f7cabcc48ff8ca007298d1c3d78)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:42.757837 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dc4114-e1fb-4b8a-87f4-ef38e1d5a2d4 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:42.759713 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 206/401] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:44:42 2022] GET /identity/v3/projects/92337f7cabcc48ff8ca007298d1c3d78 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:42.765819 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dc4114-e1fb-4b8a-87f4-ef38e1d5a2d4 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:42.800970 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 196/402] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:44:42.802334 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f799d9-cf54-4c17-8453-27c7ded463b6 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:42.802620 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f799d9-cf54-4c17-8453-27c7ded463b6 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:42.802896 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f799d9-cf54-4c17-8453-27c7ded463b6 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:42.804700 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 197/403] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:42.810931 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8dd2d-6de3-4e00-808c-1198f6dac180 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:42.811211 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8dd2d-6de3-4e00-808c-1198f6dac180 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:42.811488 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8dd2d-6de3-4e00-808c-1198f6dac180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:42.836843 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-19a8dd2d-6de3-4e00-808c-1198f6dac180 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:42.837998 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-19a8dd2d-6de3-4e00-808c-1198f6dac180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:42.905080 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dc4114-e1fb-4b8a-87f4-ef38e1d5a2d4 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:42.919712 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 198/404] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:44:42.929649 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a8dd2d-6de3-4e00-808c-1198f6dac180 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:42.945425 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed134e32-9be9-476f-8a12-c9171c68257f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:42.946967 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed134e32-9be9-476f-8a12-c9171c68257f None admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:42.947426 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed134e32-9be9-476f-8a12-c9171c68257f None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:42.947875 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed134e32-9be9-476f-8a12-c9171c68257f None admin] PATH_INFO: `/v3/projects/92337f7cabcc48ff8ca007298d1c3d78/groups/94e1fd9f935349cd9d51c38cabcba69d/roles/d328f8040f2247d083309d839d2ff947` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:42.952494 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed134e32-9be9-476f-8a12-c9171c68257f None admin] RBAC: Authorizing `identity:create_grant(project_id=92337f7cabcc48ff8ca007298d1c3d78, group_id=94e1fd9f935349cd9d51c38cabcba69d, role_id=d328f8040f2247d083309d839d2ff947)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:42.955411 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed134e32-9be9-476f-8a12-c9171c68257f None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:42.980644 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 207/405] 10.222.0.23 () {58 vars in 1442 bytes} [Fri May 6 14:44:42 2022] PUT /identity/v3/projects/92337f7cabcc48ff8ca007298d1c3d78/groups/94e1fd9f935349cd9d51c38cabcba69d/roles/d328f8040f2247d083309d839d2ff947 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:44:42.980644 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed134e32-9be9-476f-8a12-c9171c68257f None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:43.113376 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a8dd2d-6de3-4e00-808c-1198f6dac180 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:43.172814 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22664aa1-3df4-445b-b210-9aa631109b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:43.172814 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22664aa1-3df4-445b-b210-9aa631109b01 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:43.172814 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22664aa1-3df4-445b-b210-9aa631109b01 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:43.172814 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22664aa1-3df4-445b-b210-9aa631109b01 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:43.193139 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22664aa1-3df4-445b-b210-9aa631109b01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:43.202083 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-22664aa1-3df4-445b-b210-9aa631109b01 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 06 14:44:43.202083 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:43.202083 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:43.202083 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:43.202083 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:43.202083 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:43.202083 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:43.202083 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:43.202083 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:43.202083 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:43.202083 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:43.202083 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:43.202083 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:43.202083 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:43.202083 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:43.202083 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:43.203018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:43.203018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:43.203018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:43.203018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:43.203018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:43.203018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:43.203018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:43.203018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:43.203018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:43.203018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:43.203018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:43.203018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:44:43.203018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:43.203018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:43.203018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:43.203018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:43.203836 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:43.203836 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:43.203836 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:43.203836 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:43.203836 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:43.203836 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:43.203836 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:43.203836 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:43.203836 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:43.203836 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:43.203836 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 06 14:44:43.203836 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:44:43.206434 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 199/406] 10.222.0.23 () {56 vars in 1114 bytes} [Fri May 6 14:44:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:43.207238 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed134e32-9be9-476f-8a12-c9171c68257f None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:43.212138 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22664aa1-3df4-445b-b210-9aa631109b01 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:43.246303 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86da4f1-2849-4671-b300-876309fb7b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:43.247079 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86da4f1-2849-4671-b300-876309fb7b1c None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:43.247423 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86da4f1-2849-4671-b300-876309fb7b1c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:43.247758 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86da4f1-2849-4671-b300-876309fb7b1c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:43.261423 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86da4f1-2849-4671-b300-876309fb7b1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:43.271530 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-d86da4f1-2849-4671-b300-876309fb7b1c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:44:43.271530 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:43.271530 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:43.271530 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:43.271530 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:43.271530 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:43.271530 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:43.271530 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:43.271530 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:43.271530 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:43.271530 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:43.271530 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:43.271530 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:43.271530 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:43.271530 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:43.271530 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:43.272500 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:43.272500 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:43.272500 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:43.272500 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:43.272500 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:43.272500 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:43.272500 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:43.272500 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:43.272500 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:43.272500 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:43.272500 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:43.272500 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:44:43.272500 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:43.272500 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:43.272500 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:43.272500 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:43.273359 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:43.273359 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:43.273359 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:43.273359 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:43.273359 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:43.273359 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:43.273359 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:43.273359 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:43.273359 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:43.273359 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:43.273359 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:44:43.273359 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:44:43.273359 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 208/407] 10.222.0.23 () {56 vars in 1096 bytes} [Fri May 6 14:44:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:43.280359 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86da4f1-2849-4671-b300-876309fb7b1c None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:43.324692 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22664aa1-3df4-445b-b210-9aa631109b01 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:43.359588 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b226017-aea3-4d36-bcae-3608620d5634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:43.360669 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b226017-aea3-4d36-bcae-3608620d5634 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:43.361152 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b226017-aea3-4d36-bcae-3608620d5634 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:43.361624 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b226017-aea3-4d36-bcae-3608620d5634 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:43.362640 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b226017-aea3-4d36-bcae-3608620d5634 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:43.363151 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b226017-aea3-4d36-bcae-3608620d5634 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:43.364828 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b226017-aea3-4d36-bcae-3608620d5634 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:43.374282 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 200/408] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:44:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:43.381547 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b226017-aea3-4d36-bcae-3608620d5634 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:43.394917 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86da4f1-2849-4671-b300-876309fb7b1c None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:43.429462 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ceb83-a75c-4c62-b813-e327125f2d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:43.430043 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ceb83-a75c-4c62-b813-e327125f2d44 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:43.430323 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ceb83-a75c-4c62-b813-e327125f2d44 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:43.430596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ceb83-a75c-4c62-b813-e327125f2d44 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:43.431331 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ceb83-a75c-4c62-b813-e327125f2d44 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:43.431634 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ceb83-a75c-4c62-b813-e327125f2d44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:43.433048 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ceb83-a75c-4c62-b813-e327125f2d44 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:43.448175 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 209/409] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:44:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:43.449834 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ceb83-a75c-4c62-b813-e327125f2d44 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:43.568492 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ceb83-a75c-4c62-b813-e327125f2d44 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:43.588291 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b226017-aea3-4d36-bcae-3608620d5634 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:43.604044 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f094a-42be-4898-b784-d3a6a0b25443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:43.604945 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f094a-42be-4898-b784-d3a6a0b25443 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:43.605229 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f094a-42be-4898-b784-d3a6a0b25443 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:43.605517 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f094a-42be-4898-b784-d3a6a0b25443 None admin] PATH_INFO: `/v3/groups/e4a546bfd06d45958a194d886a8b846e` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:43.607166 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098f094a-42be-4898-b784-d3a6a0b25443 None admin] RBAC: Authorizing `identity:get_group(group_id=e4a546bfd06d45958a194d886a8b846e)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:43.608760 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098f094a-42be-4898-b784-d3a6a0b25443 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:43.615511 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 210/410] 10.222.0.23 () {56 vars in 1180 bytes} [Fri May 6 14:44:43 2022] GET /identity/v3/groups/e4a546bfd06d45958a194d886a8b846e => generated 229 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:43.617412 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f094a-42be-4898-b784-d3a6a0b25443 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:43.685428 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda8725e-8d5a-47d4-93fd-47aa1d35eed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:43.686227 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8725e-8d5a-47d4-93fd-47aa1d35eed6 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:43.686516 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8725e-8d5a-47d4-93fd-47aa1d35eed6 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:43.686805 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8725e-8d5a-47d4-93fd-47aa1d35eed6 None admin] PATH_INFO: `/v3/groups/94e1fd9f935349cd9d51c38cabcba69d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:43.688052 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda8725e-8d5a-47d4-93fd-47aa1d35eed6 None admin] RBAC: Authorizing `identity:get_group(group_id=94e1fd9f935349cd9d51c38cabcba69d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:43.689533 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda8725e-8d5a-47d4-93fd-47aa1d35eed6 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:43.691468 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 201/411] 10.222.0.23 () {56 vars in 1180 bytes} [Fri May 6 14:44:43 2022] GET /identity/v3/groups/94e1fd9f935349cd9d51c38cabcba69d => generated 226 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:43.697694 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda8725e-8d5a-47d4-93fd-47aa1d35eed6 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:43.728468 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f094a-42be-4898-b784-d3a6a0b25443 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:43.813375 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754c1203-3e45-459e-be0e-804d5a2a6482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:43.814254 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c1203-3e45-459e-be0e-804d5a2a6482 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:43.814540 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c1203-3e45-459e-be0e-804d5a2a6482 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:43.814815 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c1203-3e45-459e-be0e-804d5a2a6482 None admin] PATH_INFO: `/v3/projects/ccc1ae27e989468199e57e6588faed16` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:43.816082 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754c1203-3e45-459e-be0e-804d5a2a6482 None admin] RBAC: Authorizing `identity:get_project(project_id=ccc1ae27e989468199e57e6588faed16)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:43.817561 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754c1203-3e45-459e-be0e-804d5a2a6482 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:43.820161 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 211/412] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:44:43 2022] GET /identity/v3/projects/ccc1ae27e989468199e57e6588faed16 => generated 344 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:43.833512 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754c1203-3e45-459e-be0e-804d5a2a6482 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:43.868936 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda8725e-8d5a-47d4-93fd-47aa1d35eed6 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:43.962562 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754c1203-3e45-459e-be0e-804d5a2a6482 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:43.983900 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d535f-8060-4ece-ad3d-b844df76a27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:43.985376 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d535f-8060-4ece-ad3d-b844df76a27f None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:43.985851 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d535f-8060-4ece-ad3d-b844df76a27f None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:43.986315 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d535f-8060-4ece-ad3d-b844df76a27f None admin] PATH_INFO: `/v3/projects/643972018eac4175bcd3d9778028e93a` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:43.993994 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24d535f-8060-4ece-ad3d-b844df76a27f None admin] RBAC: Authorizing `identity:get_project(project_id=643972018eac4175bcd3d9778028e93a)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:43.996459 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24d535f-8060-4ece-ad3d-b844df76a27f None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:44.000047 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 202/413] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:44:43 2022] GET /identity/v3/projects/643972018eac4175bcd3d9778028e93a => generated 302 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:44.005557 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d535f-8060-4ece-ad3d-b844df76a27f None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:44.011359 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f24528-82b4-4d23-9b58-97e226ab5397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:44.012879 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f24528-82b4-4d23-9b58-97e226ab5397 None admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:44.013356 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f24528-82b4-4d23-9b58-97e226ab5397 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:44.013823 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f24528-82b4-4d23-9b58-97e226ab5397 None admin] PATH_INFO: `/v3/projects/ccc1ae27e989468199e57e6588faed16/groups/e4a546bfd06d45958a194d886a8b846e/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:44.021459 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f24528-82b4-4d23-9b58-97e226ab5397 None admin] RBAC: Authorizing `identity:create_grant(project_id=ccc1ae27e989468199e57e6588faed16, group_id=e4a546bfd06d45958a194d886a8b846e, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:44.022956 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f24528-82b4-4d23-9b58-97e226ab5397 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:44.051912 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 212/414] 10.222.0.23 () {58 vars in 1442 bytes} [Fri May 6 14:44:43 2022] PUT /identity/v3/projects/ccc1ae27e989468199e57e6588faed16/groups/e4a546bfd06d45958a194d886a8b846e/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 14:44:44.194487 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d535f-8060-4ece-ad3d-b844df76a27f None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:44.247406 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626558e5-2db5-4e62-bbf0-62386e65970a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:44.248305 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626558e5-2db5-4e62-bbf0-62386e65970a None admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:44.248588 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626558e5-2db5-4e62-bbf0-62386e65970a None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:44.248883 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626558e5-2db5-4e62-bbf0-62386e65970a None admin] PATH_INFO: `/v3/projects/643972018eac4175bcd3d9778028e93a/groups/94e1fd9f935349cd9d51c38cabcba69d/roles/d328f8040f2247d083309d839d2ff947` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:44.251240 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626558e5-2db5-4e62-bbf0-62386e65970a None admin] RBAC: Authorizing `identity:create_grant(project_id=643972018eac4175bcd3d9778028e93a, group_id=94e1fd9f935349cd9d51c38cabcba69d, role_id=d328f8040f2247d083309d839d2ff947)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:44.252724 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626558e5-2db5-4e62-bbf0-62386e65970a None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:44.269281 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 203/415] 10.222.0.23 () {58 vars in 1442 bytes} [Fri May 6 14:44:44 2022] PUT /identity/v3/projects/643972018eac4175bcd3d9778028e93a/groups/94e1fd9f935349cd9d51c38cabcba69d/roles/d328f8040f2247d083309d839d2ff947 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 14:44:44.356462 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338dda86-9da3-40e0-afce-30f42a31de18 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:44.356791 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338dda86-9da3-40e0-afce-30f42a31de18 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:44.357066 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338dda86-9da3-40e0-afce-30f42a31de18 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:44.358727 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 213/416] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:44.364538 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f1bf92-9e6b-425b-8049-d55e1c6904b7 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:44.364850 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f1bf92-9e6b-425b-8049-d55e1c6904b7 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:44.365126 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f1bf92-9e6b-425b-8049-d55e1c6904b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:44.390601 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-d6f1bf92-9e6b-425b-8049-d55e1c6904b7 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:44.391806 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d6f1bf92-9e6b-425b-8049-d55e1c6904b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:44.484535 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 204/417] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:44:44.509210 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338dda86-9da3-40e0-afce-30f42a31de18 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:44.527107 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b820e-8a77-4d7e-ad8e-d897cd0d856d None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:44.527426 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b820e-8a77-4d7e-ad8e-d897cd0d856d None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:44.527669 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b820e-8a77-4d7e-ad8e-d897cd0d856d None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:44.529375 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 205/418] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:44.535358 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b63ad0-d907-482c-81eb-659e32bfc797 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:44.535653 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b63ad0-d907-482c-81eb-659e32bfc797 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:44.535929 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b63ad0-d907-482c-81eb-659e32bfc797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:44.552983 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-81b63ad0-d907-482c-81eb-659e32bfc797 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:44.554764 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-81b63ad0-d907-482c-81eb-659e32bfc797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:44.631389 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 206/419] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 06 14:44:44.656143 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b63ad0-d907-482c-81eb-659e32bfc797 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:44.689021 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338dda86-9da3-40e0-afce-30f42a31de18 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:44.752126 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b3baf-1e15-48e8-84fc-7782708feef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:44.752957 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b3baf-1e15-48e8-84fc-7782708feef9 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:44.753237 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b3baf-1e15-48e8-84fc-7782708feef9 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:44.753507 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b3baf-1e15-48e8-84fc-7782708feef9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:44.767502 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1b3baf-1e15-48e8-84fc-7782708feef9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:44.779303 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-ac1b3baf-1e15-48e8-84fc-7782708feef9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 06 14:44:44.779303 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:44.779303 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:44.779303 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:44.779303 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:44.779303 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:44.779303 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:44.779303 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:44.779303 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:44.779303 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:44.779303 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:44.779303 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:44.779303 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:44.779303 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:44.779303 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:44.779303 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:44.780232 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:44.780232 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:44.780232 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:44.780232 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:44.780232 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:44.780232 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:44.780232 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:44.780232 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:44.780232 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:44.780232 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:44.780232 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:44.780232 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:44:44.780232 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:44.780232 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:44.780232 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:44.780232 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:44.781346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:44.781346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:44.781346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:44.781346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:44.781346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:44.781346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:44.781346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:44.781346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:44.781346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:44.781346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:44.781346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 06 14:44:44.781346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:44:44.781346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 214/420] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:44:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:44.791188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b3baf-1e15-48e8-84fc-7782708feef9 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:44.844455 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b63ad0-d907-482c-81eb-659e32bfc797 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:44.910711 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dbc538-0f20-4708-9df8-f191eb412f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:44.911485 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dbc538-0f20-4708-9df8-f191eb412f01 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:44.911767 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dbc538-0f20-4708-9df8-f191eb412f01 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:44.912040 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dbc538-0f20-4708-9df8-f191eb412f01 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:44.933747 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dbc538-0f20-4708-9df8-f191eb412f01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:44.941520 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-f9dbc538-0f20-4708-9df8-f191eb412f01 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 06 14:44:44.941520 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:44.941520 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:44.941520 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:44.941520 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:44.941520 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:44.941520 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:44.941520 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:44.941520 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:44.941520 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:44.941520 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:44.941520 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:44.941520 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:44.941520 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:44.941520 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:44.941520 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:44.942398 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:44.942398 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:44.942398 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:44.942398 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:44.942398 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:44.942398 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:44.942398 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:44.942398 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:44.942398 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:44.942398 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:44.942398 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:44.942398 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:44:44.942398 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:44.942398 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:44.942398 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:44.942398 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:44.943183 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:44.943183 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:44.943183 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:44.943183 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:44.943183 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:44.943183 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:44.943183 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:44.943183 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:44.943183 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:44.943183 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:44.943183 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 06 14:44:44.943183 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:44:44.943183 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 207/421] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:44:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:44.949433 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dbc538-0f20-4708-9df8-f191eb412f01 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:44.960474 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b3baf-1e15-48e8-84fc-7782708feef9 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:45.029977 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86055b6e-0f0a-44b0-868c-db16c7e7b7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:45.030595 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86055b6e-0f0a-44b0-868c-db16c7e7b7c7 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:45.031115 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86055b6e-0f0a-44b0-868c-db16c7e7b7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:45.031394 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86055b6e-0f0a-44b0-868c-db16c7e7b7c7 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:45.032144 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86055b6e-0f0a-44b0-868c-db16c7e7b7c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:45.032450 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86055b6e-0f0a-44b0-868c-db16c7e7b7c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:45.033846 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86055b6e-0f0a-44b0-868c-db16c7e7b7c7 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:45.045517 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 215/422] 10.222.0.23 () {56 vars in 1101 bytes} [Fri May 6 14:44:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:45.051596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86055b6e-0f0a-44b0-868c-db16c7e7b7c7 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:45.160222 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86055b6e-0f0a-44b0-868c-db16c7e7b7c7 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:45.174795 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dbc538-0f20-4708-9df8-f191eb412f01 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:45.196976 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4d70d-11aa-4415-a870-c04e10403c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:45.197641 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4d70d-11aa-4415-a870-c04e10403c4c None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:45.197930 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4d70d-11aa-4415-a870-c04e10403c4c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:45.198212 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4d70d-11aa-4415-a870-c04e10403c4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:45.268591 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc4d70d-11aa-4415-a870-c04e10403c4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:45.270225 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc4d70d-11aa-4415-a870-c04e10403c4c None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:45.282296 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db2ac5a-db31-498e-9324-61023bcb012d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:45.282905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2ac5a-db31-498e-9324-61023bcb012d None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:45.283186 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2ac5a-db31-498e-9324-61023bcb012d None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:45.283463 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2ac5a-db31-498e-9324-61023bcb012d None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:45.284213 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db2ac5a-db31-498e-9324-61023bcb012d None admin] RBAC: Adding query filter params (name=member) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:45.284516 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db2ac5a-db31-498e-9324-61023bcb012d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:45.285920 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db2ac5a-db31-498e-9324-61023bcb012d None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:45.304373 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 208/423] 10.222.0.23 () {56 vars in 1101 bytes} [Fri May 6 14:44:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:45.313649 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db2ac5a-db31-498e-9324-61023bcb012d None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:45.372217 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 216/424] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:44:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 06 14:44:45.378750 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4d70d-11aa-4415-a870-c04e10403c4c None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:45.525825 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db2ac5a-db31-498e-9324-61023bcb012d None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:45.568851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4d70d-11aa-4415-a870-c04e10403c4c None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:45.603813 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc229e09-584f-4601-8229-fc2a5f37b629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:45.604789 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc229e09-584f-4601-8229-fc2a5f37b629 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:45.605222 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc229e09-584f-4601-8229-fc2a5f37b629 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:45.605648 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc229e09-584f-4601-8229-fc2a5f37b629 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:45.664295 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d03abc0-8f88-4f44-bc1d-10beac8384c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:45.665601 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03abc0-8f88-4f44-bc1d-10beac8384c6 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:45.666008 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03abc0-8f88-4f44-bc1d-10beac8384c6 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:45.666409 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03abc0-8f88-4f44-bc1d-10beac8384c6 None admin] PATH_INFO: `/v3/projects/92337f7cabcc48ff8ca007298d1c3d78` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:45.668995 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d03abc0-8f88-4f44-bc1d-10beac8384c6 None admin] RBAC: Authorizing `identity:get_project(project_id=92337f7cabcc48ff8ca007298d1c3d78)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:45.672516 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d03abc0-8f88-4f44-bc1d-10beac8384c6 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:45.675140 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 217/425] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:44:45 2022] GET /identity/v3/projects/92337f7cabcc48ff8ca007298d1c3d78 => generated 298 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:45.680945 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d03abc0-8f88-4f44-bc1d-10beac8384c6 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:45.781881 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc229e09-584f-4601-8229-fc2a5f37b629 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:45.783353 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc229e09-584f-4601-8229-fc2a5f37b629 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:45.814345 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d03abc0-8f88-4f44-bc1d-10beac8384c6 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:45.858590 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6423090-0d16-4b61-81f3-7199ecb9d62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:45.859397 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6423090-0d16-4b61-81f3-7199ecb9d62c None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:45.859682 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6423090-0d16-4b61-81f3-7199ecb9d62c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:45.859957 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6423090-0d16-4b61-81f3-7199ecb9d62c None admin] PATH_INFO: `/v3/groups/94e1fd9f935349cd9d51c38cabcba69d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:45.861167 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6423090-0d16-4b61-81f3-7199ecb9d62c None admin] RBAC: Authorizing `identity:get_group(group_id=94e1fd9f935349cd9d51c38cabcba69d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:45.862594 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6423090-0d16-4b61-81f3-7199ecb9d62c None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:45.865150 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 218/426] 10.222.0.23 () {56 vars in 1180 bytes} [Fri May 6 14:44:45 2022] GET /identity/v3/groups/94e1fd9f935349cd9d51c38cabcba69d => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:45.866605 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1bd57-6bff-424d-b2db-d0b0dcea7305 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:45.866605 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1bd57-6bff-424d-b2db-d0b0dcea7305 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:45.867529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1bd57-6bff-424d-b2db-d0b0dcea7305 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:45.868804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 219/427] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:45.873812 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1bd57-6bff-424d-b2db-d0b0dcea7305 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:45.896021 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 209/428] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:44:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 583 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 06 14:44:45.902616 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858ca36-90e6-495f-979e-e2b38c58743b None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:45.903036 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858ca36-90e6-495f-979e-e2b38c58743b None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:45.903385 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858ca36-90e6-495f-979e-e2b38c58743b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:45.921297 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-e858ca36-90e6-495f-979e-e2b38c58743b None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:45.922568 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-e858ca36-90e6-495f-979e-e2b38c58743b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:45.976664 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1bd57-6bff-424d-b2db-d0b0dcea7305 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:45.991740 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 210/429] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:44:45.992583 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e858ca36-90e6-495f-979e-e2b38c58743b None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:46.034940 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b8e3d5-a2fc-43f7-a535-825a01f6b00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:46.035605 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8e3d5-a2fc-43f7-a535-825a01f6b00d None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:46.035884 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8e3d5-a2fc-43f7-a535-825a01f6b00d None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:46.036156 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8e3d5-a2fc-43f7-a535-825a01f6b00d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:46.037003 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b8e3d5-a2fc-43f7-a535-825a01f6b00d None admin] RBAC: Adding query filter params (group.id=94e1fd9f935349cd9d51c38cabcba69d, role.id=5f76e2e820b64d14b893c679427f4bed, scope.project.id=92337f7cabcc48ff8ca007298d1c3d78) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:46.037313 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b8e3d5-a2fc-43f7-a535-825a01f6b00d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:46.039380 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b8e3d5-a2fc-43f7-a535-825a01f6b00d None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:46.045985 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 220/430] 10.222.0.23 () {56 vars in 1376 bytes} [Fri May 6 14:44:45 2022] GET /identity/v3/role_assignments?group.id=94e1fd9f935349cd9d51c38cabcba69d&scope.project.id=92337f7cabcc48ff8ca007298d1c3d78&role.id=5f76e2e820b64d14b893c679427f4bed => generated 611 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:46.046839 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b8e3d5-a2fc-43f7-a535-825a01f6b00d None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:46.107581 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e858ca36-90e6-495f-979e-e2b38c58743b None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:46.143663 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb25e8-798a-4822-85a0-1cd572744ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:46.144450 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb25e8-798a-4822-85a0-1cd572744ec1 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:46.144762 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb25e8-798a-4822-85a0-1cd572744ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:46.145042 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb25e8-798a-4822-85a0-1cd572744ec1 None admin] PATH_INFO: `/v3/projects/643972018eac4175bcd3d9778028e93a` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:46.146219 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eb25e8-798a-4822-85a0-1cd572744ec1 None admin] RBAC: Authorizing `identity:get_project(project_id=643972018eac4175bcd3d9778028e93a)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:46.148725 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eb25e8-798a-4822-85a0-1cd572744ec1 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:46.160373 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b8e3d5-a2fc-43f7-a535-825a01f6b00d None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:46.165883 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 211/431] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:44:45 2022] GET /identity/v3/projects/643972018eac4175bcd3d9778028e93a => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:46.166953 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb25e8-798a-4822-85a0-1cd572744ec1 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:46.274866 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3315d-88fb-4602-8d8f-79dd3f841536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:46.275074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb25e8-798a-4822-85a0-1cd572744ec1 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:46.276181 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3315d-88fb-4602-8d8f-79dd3f841536 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:46.276726 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3315d-88fb-4602-8d8f-79dd3f841536 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:46.277212 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3315d-88fb-4602-8d8f-79dd3f841536 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:46.296262 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f3315d-88fb-4602-8d8f-79dd3f841536 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:46.304340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-17f3315d-88fb-4602-8d8f-79dd3f841536 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 06 14:44:46.304340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:46.304340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:46.304340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:46.304340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:46.304340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:46.304340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:46.304340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:46.304340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:46.304340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:46.304340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:46.304340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:46.304340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:46.304340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:46.304340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:46.304340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:46.305263 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:46.305263 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:46.305263 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:46.305263 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:46.305263 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:46.305263 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:46.305263 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:46.305263 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:46.305263 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:46.305263 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:46.305263 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:46.305263 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:44:46.305263 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:46.305263 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:46.305263 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:46.305263 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:46.306051 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:46.306051 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:46.306051 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:46.306051 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:46.306051 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:46.306051 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:46.306051 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:46.306051 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:46.306051 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:46.306051 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:46.306051 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 06 14:44:46.306051 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:44:46.306051 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 221/432] 10.222.0.23 () {56 vars in 1114 bytes} [Fri May 6 14:44:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:46.311554 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3315d-88fb-4602-8d8f-79dd3f841536 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:46.312163 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0387498f-3219-4f34-a40c-4717e6bc02e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:46.313074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0387498f-3219-4f34-a40c-4717e6bc02e6 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:46.313412 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0387498f-3219-4f34-a40c-4717e6bc02e6 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:46.313685 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0387498f-3219-4f34-a40c-4717e6bc02e6 None admin] PATH_INFO: `/v3/groups/94e1fd9f935349cd9d51c38cabcba69d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:46.314912 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0387498f-3219-4f34-a40c-4717e6bc02e6 None admin] RBAC: Authorizing `identity:get_group(group_id=94e1fd9f935349cd9d51c38cabcba69d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:46.316342 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0387498f-3219-4f34-a40c-4717e6bc02e6 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:46.318683 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 212/433] 10.222.0.23 () {56 vars in 1180 bytes} [Fri May 6 14:44:46 2022] GET /identity/v3/groups/94e1fd9f935349cd9d51c38cabcba69d => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:46.325190 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0387498f-3219-4f34-a40c-4717e6bc02e6 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:46.426265 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0387498f-3219-4f34-a40c-4717e6bc02e6 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:46.426839 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3315d-88fb-4602-8d8f-79dd3f841536 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:46.463331 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e8cbb-ba36-4652-b1e9-c886a2fb5fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:46.464005 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e8cbb-ba36-4652-b1e9-c886a2fb5fb3 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:46.464294 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e8cbb-ba36-4652-b1e9-c886a2fb5fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:46.464573 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e8cbb-ba36-4652-b1e9-c886a2fb5fb3 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:46.465355 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e8cbb-ba36-4652-b1e9-c886a2fb5fb3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:46.465665 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e8cbb-ba36-4652-b1e9-c886a2fb5fb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:46.467055 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e8cbb-ba36-4652-b1e9-c886a2fb5fb3 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:46.473125 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d835163f-b4bb-47a3-ae90-40ecbcc9dda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:46.473838 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d835163f-b4bb-47a3-ae90-40ecbcc9dda5 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:46.474121 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d835163f-b4bb-47a3-ae90-40ecbcc9dda5 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:46.474394 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d835163f-b4bb-47a3-ae90-40ecbcc9dda5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:46.475216 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d835163f-b4bb-47a3-ae90-40ecbcc9dda5 None admin] RBAC: Adding query filter params (group.id=94e1fd9f935349cd9d51c38cabcba69d, role.id=5f76e2e820b64d14b893c679427f4bed, scope.project.id=643972018eac4175bcd3d9778028e93a) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:46.475676 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d835163f-b4bb-47a3-ae90-40ecbcc9dda5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:46.475984 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 222/434] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:44:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:46.477194 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d835163f-b4bb-47a3-ae90-40ecbcc9dda5 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:46.481234 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e8cbb-ba36-4652-b1e9-c886a2fb5fb3 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:46.486182 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 213/435] 10.222.0.23 () {56 vars in 1376 bytes} [Fri May 6 14:44:46 2022] GET /identity/v3/role_assignments?group.id=94e1fd9f935349cd9d51c38cabcba69d&scope.project.id=643972018eac4175bcd3d9778028e93a&role.id=5f76e2e820b64d14b893c679427f4bed => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:46.596190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e8cbb-ba36-4652-b1e9-c886a2fb5fb3 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:46.633442 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a63ae7a-ab4e-4b9f-8cba-305e876ad951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:46.634181 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63ae7a-ab4e-4b9f-8cba-305e876ad951 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:46.634468 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63ae7a-ab4e-4b9f-8cba-305e876ad951 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:46.634747 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63ae7a-ab4e-4b9f-8cba-305e876ad951 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:46.708897 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a63ae7a-ab4e-4b9f-8cba-305e876ad951 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:46.710355 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a63ae7a-ab4e-4b9f-8cba-305e876ad951 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:46.814493 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 223/436] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:44:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 06 14:44:46.820412 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d835163f-b4bb-47a3-ae90-40ecbcc9dda5 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:46.924563 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4864c813-8d7e-48df-b683-eb77bf52e252 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:46.925108 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4864c813-8d7e-48df-b683-eb77bf52e252 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:46.925389 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4864c813-8d7e-48df-b683-eb77bf52e252 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:46.926875 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 224/437] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:46.934556 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eccc914-3ef5-4fa7-be4e-e068b9bfb77b None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:46.934851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eccc914-3ef5-4fa7-be4e-e068b9bfb77b None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:46.935150 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eccc914-3ef5-4fa7-be4e-e068b9bfb77b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:46.938897 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d835163f-b4bb-47a3-ae90-40ecbcc9dda5 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:46.952990 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-7eccc914-3ef5-4fa7-be4e-e068b9bfb77b None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:46.954469 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-7eccc914-3ef5-4fa7-be4e-e068b9bfb77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:46.972091 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cd674b-c076-4cbb-a0ce-bd99c71c67e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:46.972922 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd674b-c076-4cbb-a0ce-bd99c71c67e4 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:46.973212 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd674b-c076-4cbb-a0ce-bd99c71c67e4 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:46.973470 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd674b-c076-4cbb-a0ce-bd99c71c67e4 None admin] PATH_INFO: `/v3/projects/92337f7cabcc48ff8ca007298d1c3d78` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:46.974638 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cd674b-c076-4cbb-a0ce-bd99c71c67e4 None admin] RBAC: Authorizing `identity:get_project(project_id=92337f7cabcc48ff8ca007298d1c3d78)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:46.976086 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cd674b-c076-4cbb-a0ce-bd99c71c67e4 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:46.977778 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 214/438] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:44:46 2022] GET /identity/v3/projects/92337f7cabcc48ff8ca007298d1c3d78 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:46.983253 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cd674b-c076-4cbb-a0ce-bd99c71c67e4 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:47.026015 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 225/439] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 06 14:44:47.049799 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a9611-875e-45db-a504-4d0eec854d3c None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:47.050391 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a9611-875e-45db-a504-4d0eec854d3c None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:47.050391 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a9611-875e-45db-a504-4d0eec854d3c None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:47.051691 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 226/440] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:44:47.052387 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353a9611-875e-45db-a504-4d0eec854d3c None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:47.081627 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cd674b-c076-4cbb-a0ce-bd99c71c67e4 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:47.113495 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9ced91-0d0a-4ba6-bde4-c1657619517c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:47.114341 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ced91-0d0a-4ba6-bde4-c1657619517c None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:47.114618 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ced91-0d0a-4ba6-bde4-c1657619517c None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:47.114890 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ced91-0d0a-4ba6-bde4-c1657619517c None admin] PATH_INFO: `/v3/groups/94e1fd9f935349cd9d51c38cabcba69d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:47.116130 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9ced91-0d0a-4ba6-bde4-c1657619517c None admin] RBAC: Authorizing `identity:get_group(group_id=94e1fd9f935349cd9d51c38cabcba69d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:47.117582 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9ced91-0d0a-4ba6-bde4-c1657619517c None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:47.119180 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 215/441] 10.222.0.23 () {56 vars in 1180 bytes} [Fri May 6 14:44:46 2022] GET /identity/v3/groups/94e1fd9f935349cd9d51c38cabcba69d => generated 226 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:47.121853 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48875ce-3cd6-40cc-b28d-97e9bb3b1fb5 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:47.122143 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48875ce-3cd6-40cc-b28d-97e9bb3b1fb5 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:47.122429 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48875ce-3cd6-40cc-b28d-97e9bb3b1fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:47.138856 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-e48875ce-3cd6-40cc-b28d-97e9bb3b1fb5 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:47.140039 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-e48875ce-3cd6-40cc-b28d-97e9bb3b1fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:47.153810 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353a9611-875e-45db-a504-4d0eec854d3c None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:47.187046 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6908a03-f145-45dc-8686-1bb0b5cad31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:47.187834 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6908a03-f145-45dc-8686-1bb0b5cad31e None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:47.188111 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6908a03-f145-45dc-8686-1bb0b5cad31e None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:47.188382 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6908a03-f145-45dc-8686-1bb0b5cad31e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:47.199392 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6908a03-f145-45dc-8686-1bb0b5cad31e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:47.205254 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 216/442] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:44:47.206195 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48875ce-3cd6-40cc-b28d-97e9bb3b1fb5 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:47.207767 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-b6908a03-f145-45dc-8686-1bb0b5cad31e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:44:47.207767 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:47.207767 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:47.207767 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:47.207767 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:47.207767 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:47.207767 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:47.207767 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:47.207767 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:47.207767 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:47.207767 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:47.207767 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:47.207767 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:47.207767 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:47.207767 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:47.207767 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:47.208718 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:47.208718 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:47.208718 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:47.208718 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:47.208718 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:47.208718 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:47.208718 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:47.208718 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:47.208718 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:47.208718 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:47.208718 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:47.208718 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:44:47.208718 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:47.208718 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:47.208718 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:47.208718 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:47.209529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:47.209529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:47.209529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:47.209529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:47.209529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:47.209529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:47.209529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:47.209529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:47.209529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:47.209529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:47.209529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:44:47.209529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:44:47.209529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 227/443] 10.222.0.23 () {56 vars in 1096 bytes} [Fri May 6 14:44:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:47.215573 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6908a03-f145-45dc-8686-1bb0b5cad31e None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:47.303366 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48875ce-3cd6-40cc-b28d-97e9bb3b1fb5 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:47.309481 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6908a03-f145-45dc-8686-1bb0b5cad31e None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:47.336171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231968b3-c116-48b0-9708-9e858806993b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:47.336845 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231968b3-c116-48b0-9708-9e858806993b None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:47.337126 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231968b3-c116-48b0-9708-9e858806993b None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:47.337397 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231968b3-c116-48b0-9708-9e858806993b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:47.338217 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231968b3-c116-48b0-9708-9e858806993b None admin] RBAC: Adding query filter params (group.id=94e1fd9f935349cd9d51c38cabcba69d, role.id=d328f8040f2247d083309d839d2ff947, scope.project.id=92337f7cabcc48ff8ca007298d1c3d78) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:47.338522 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231968b3-c116-48b0-9708-9e858806993b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:47.339942 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231968b3-c116-48b0-9708-9e858806993b None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:47.341100 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5a283-34af-41cf-a1df-e66fecddcaf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:47.341758 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5a283-34af-41cf-a1df-e66fecddcaf9 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:47.342036 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5a283-34af-41cf-a1df-e66fecddcaf9 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:47.342309 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5a283-34af-41cf-a1df-e66fecddcaf9 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:47.343037 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad5a283-34af-41cf-a1df-e66fecddcaf9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:47.343339 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad5a283-34af-41cf-a1df-e66fecddcaf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:47.344721 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad5a283-34af-41cf-a1df-e66fecddcaf9 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:47.345624 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 217/444] 10.222.0.23 () {56 vars in 1376 bytes} [Fri May 6 14:44:47 2022] GET /identity/v3/role_assignments?group.id=94e1fd9f935349cd9d51c38cabcba69d&scope.project.id=92337f7cabcc48ff8ca007298d1c3d78&role.id=d328f8040f2247d083309d839d2ff947 => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:47.346398 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231968b3-c116-48b0-9708-9e858806993b None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:47.370288 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 228/445] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:44:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:47.376050 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5a283-34af-41cf-a1df-e66fecddcaf9 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:47.452686 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231968b3-c116-48b0-9708-9e858806993b None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:47.482498 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5a283-34af-41cf-a1df-e66fecddcaf9 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:47.489575 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc317d58-0660-4783-9e6f-28b92360bb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:47.491529 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc317d58-0660-4783-9e6f-28b92360bb6f None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:47.492349 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc317d58-0660-4783-9e6f-28b92360bb6f None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:47.492349 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc317d58-0660-4783-9e6f-28b92360bb6f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:47.507230 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc317d58-0660-4783-9e6f-28b92360bb6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:47.517000 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-fc317d58-0660-4783-9e6f-28b92360bb6f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 06 14:44:47.517000 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:47.517000 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:47.517000 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:47.517000 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:47.517000 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:47.517000 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:47.517000 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:47.517000 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:47.517000 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:47.517000 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:47.517000 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:47.517000 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:47.517000 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:47.517000 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:47.517000 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:47.517968 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:47.517968 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:47.517968 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:47.517968 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:47.517968 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:47.517968 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:47.517968 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:47.517968 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:47.517968 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:47.517968 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:47.517968 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:47.517968 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:44:47.517968 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:47.517968 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:47.517968 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:47.517968 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:47.518758 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:47.518758 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:47.518758 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:47.518758 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:47.518758 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:47.518758 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:47.518758 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:47.518758 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:47.518758 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:47.518758 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:47.518758 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 06 14:44:47.518758 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:44:47.518758 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 218/446] 10.222.0.23 () {56 vars in 1114 bytes} [Fri May 6 14:44:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:47.521404 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced98582-5536-4762-9cc9-44489216264e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:47.522066 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced98582-5536-4762-9cc9-44489216264e None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:47.522503 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced98582-5536-4762-9cc9-44489216264e None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:47.523504 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced98582-5536-4762-9cc9-44489216264e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:47.524230 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc317d58-0660-4783-9e6f-28b92360bb6f None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:47.594983 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced98582-5536-4762-9cc9-44489216264e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:47.596402 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced98582-5536-4762-9cc9-44489216264e None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:47.636577 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc317d58-0660-4783-9e6f-28b92360bb6f None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:47.672346 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb51757-a0ed-4cf0-bbad-7908d000fd24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:47.673361 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb51757-a0ed-4cf0-bbad-7908d000fd24 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:47.673646 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb51757-a0ed-4cf0-bbad-7908d000fd24 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:47.673930 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb51757-a0ed-4cf0-bbad-7908d000fd24 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:47.674666 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb51757-a0ed-4cf0-bbad-7908d000fd24 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:47.674971 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb51757-a0ed-4cf0-bbad-7908d000fd24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:47.676900 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb51757-a0ed-4cf0-bbad-7908d000fd24 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:47.685605 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 219/447] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:44:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:47.691073 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb51757-a0ed-4cf0-bbad-7908d000fd24 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:47.713737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 229/448] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:44:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 06 14:44:47.719920 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced98582-5536-4762-9cc9-44489216264e None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:47.803509 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb51757-a0ed-4cf0-bbad-7908d000fd24 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:47.830717 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced98582-5536-4762-9cc9-44489216264e None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:47.845186 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23a219-45ea-4c9d-a491-f507608b465d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:47.845779 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23a219-45ea-4c9d-a491-f507608b465d None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:47.846066 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23a219-45ea-4c9d-a491-f507608b465d None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:47.846341 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23a219-45ea-4c9d-a491-f507608b465d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:47.869985 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62c2987-39e1-41a4-a2fb-2687a1c020f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:47.871846 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c2987-39e1-41a4-a2fb-2687a1c020f8 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:47.871846 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c2987-39e1-41a4-a2fb-2687a1c020f8 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:47.871846 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c2987-39e1-41a4-a2fb-2687a1c020f8 None admin] PATH_INFO: `/v3/projects/ccc1ae27e989468199e57e6588faed16` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:47.872920 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62c2987-39e1-41a4-a2fb-2687a1c020f8 None admin] RBAC: Authorizing `identity:get_project(project_id=ccc1ae27e989468199e57e6588faed16)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:47.874343 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62c2987-39e1-41a4-a2fb-2687a1c020f8 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:47.880732 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 230/449] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:44:47 2022] GET /identity/v3/projects/ccc1ae27e989468199e57e6588faed16 => generated 344 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:47.882516 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62c2987-39e1-41a4-a2fb-2687a1c020f8 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:47.923254 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae23a219-45ea-4c9d-a491-f507608b465d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:47.924751 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae23a219-45ea-4c9d-a491-f507608b465d None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:47.977873 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62c2987-39e1-41a4-a2fb-2687a1c020f8 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:48.010367 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77da2726-05e5-4a1e-b341-7390c3dea83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:48.011219 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da2726-05e5-4a1e-b341-7390c3dea83e None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:48.011505 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da2726-05e5-4a1e-b341-7390c3dea83e None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:48.011783 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da2726-05e5-4a1e-b341-7390c3dea83e None admin] PATH_INFO: `/v3/groups/e4a546bfd06d45958a194d886a8b846e` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:48.012993 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77da2726-05e5-4a1e-b341-7390c3dea83e None admin] RBAC: Authorizing `identity:get_group(group_id=e4a546bfd06d45958a194d886a8b846e)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:48.014407 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77da2726-05e5-4a1e-b341-7390c3dea83e None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:48.016325 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 231/450] 10.222.0.23 () {56 vars in 1180 bytes} [Fri May 6 14:44:47 2022] GET /identity/v3/groups/e4a546bfd06d45958a194d886a8b846e => generated 229 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:48.023992 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 220/451] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:44:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 06 14:44:48.024151 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77da2726-05e5-4a1e-b341-7390c3dea83e None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:48.024994 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23a219-45ea-4c9d-a491-f507608b465d None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:48.115459 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77da2726-05e5-4a1e-b341-7390c3dea83e None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:48.116242 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23a219-45ea-4c9d-a491-f507608b465d None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:48.146880 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f46c45-450e-45b5-a23a-fe0f324cfe98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:48.147803 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfad3b46-23b9-4fc5-9694-720a4944d295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:48.148072 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f46c45-450e-45b5-a23a-fe0f324cfe98 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:48.148323 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f46c45-450e-45b5-a23a-fe0f324cfe98 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:48.148628 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f46c45-450e-45b5-a23a-fe0f324cfe98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:48.148758 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad3b46-23b9-4fc5-9694-720a4944d295 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:48.149043 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad3b46-23b9-4fc5-9694-720a4944d295 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:48.149311 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad3b46-23b9-4fc5-9694-720a4944d295 None admin] PATH_INFO: `/v3/projects/643972018eac4175bcd3d9778028e93a` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:48.149440 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f46c45-450e-45b5-a23a-fe0f324cfe98 None admin] RBAC: Adding query filter params (group.id=e4a546bfd06d45958a194d886a8b846e, role.id=6dcd35d1a83748759d6ef9f026077835, scope.project.id=ccc1ae27e989468199e57e6588faed16) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:48.149749 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f46c45-450e-45b5-a23a-fe0f324cfe98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:48.150466 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfad3b46-23b9-4fc5-9694-720a4944d295 None admin] RBAC: Authorizing `identity:get_project(project_id=643972018eac4175bcd3d9778028e93a)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:48.153504 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f46c45-450e-45b5-a23a-fe0f324cfe98 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:48.158284 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfad3b46-23b9-4fc5-9694-720a4944d295 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:48.158284 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 221/452] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:44:48 2022] GET /identity/v3/projects/643972018eac4175bcd3d9778028e93a => generated 302 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:48.159213 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfad3b46-23b9-4fc5-9694-720a4944d295 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:48.161609 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 232/453] 10.222.0.23 () {56 vars in 1376 bytes} [Fri May 6 14:44:48 2022] GET /identity/v3/role_assignments?group.id=e4a546bfd06d45958a194d886a8b846e&scope.project.id=ccc1ae27e989468199e57e6588faed16&role.id=6dcd35d1a83748759d6ef9f026077835 => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:48.262856 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfad3b46-23b9-4fc5-9694-720a4944d295 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:48.298566 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18242851-103d-4720-9003-9825a5b8e0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:48.299425 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18242851-103d-4720-9003-9825a5b8e0cd None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:48.299700 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18242851-103d-4720-9003-9825a5b8e0cd None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:48.299969 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18242851-103d-4720-9003-9825a5b8e0cd None admin] PATH_INFO: `/v3/groups/94e1fd9f935349cd9d51c38cabcba69d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:48.301189 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18242851-103d-4720-9003-9825a5b8e0cd None admin] RBAC: Authorizing `identity:get_group(group_id=94e1fd9f935349cd9d51c38cabcba69d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:48.302617 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18242851-103d-4720-9003-9825a5b8e0cd None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:48.305222 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 222/454] 10.222.0.23 () {56 vars in 1180 bytes} [Fri May 6 14:44:48 2022] GET /identity/v3/groups/94e1fd9f935349cd9d51c38cabcba69d => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:48.311117 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f46c45-450e-45b5-a23a-fe0f324cfe98 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:48.415358 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f46c45-450e-45b5-a23a-fe0f324cfe98 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:48.446750 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d9a928-a260-4c99-8737-0d732c728e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:48.447384 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d9a928-a260-4c99-8737-0d732c728e4c None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:48.447663 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d9a928-a260-4c99-8737-0d732c728e4c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:48.447932 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d9a928-a260-4c99-8737-0d732c728e4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:48.448769 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d9a928-a260-4c99-8737-0d732c728e4c None admin] RBAC: Adding query filter params (group.id=94e1fd9f935349cd9d51c38cabcba69d, role.id=d328f8040f2247d083309d839d2ff947, scope.project.id=643972018eac4175bcd3d9778028e93a) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:48.449102 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d9a928-a260-4c99-8737-0d732c728e4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:48.450524 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d9a928-a260-4c99-8737-0d732c728e4c None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:48.455999 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 233/455] 10.222.0.23 () {56 vars in 1376 bytes} [Fri May 6 14:44:48 2022] GET /identity/v3/role_assignments?group.id=94e1fd9f935349cd9d51c38cabcba69d&scope.project.id=643972018eac4175bcd3d9778028e93a&role.id=d328f8040f2247d083309d839d2ff947 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:51.473615 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b81d47-d4d8-4c53-86bc-a2cd11cb5f98 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:51.473615 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b81d47-d4d8-4c53-86bc-a2cd11cb5f98 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:51.473615 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b81d47-d4d8-4c53-86bc-a2cd11cb5f98 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:51.474495 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 223/456] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:51.480236 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee788f6f-c7ab-416b-983c-afd38b6a05be None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:51.481042 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee788f6f-c7ab-416b-983c-afd38b6a05be None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:51.481470 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee788f6f-c7ab-416b-983c-afd38b6a05be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:51.509408 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-ee788f6f-c7ab-416b-983c-afd38b6a05be None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:51.510938 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-ee788f6f-c7ab-416b-983c-afd38b6a05be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:51.596202 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 234/457] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 06 14:44:51.633741 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b81d47-d4d8-4c53-86bc-a2cd11cb5f98 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:51.778284 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b81d47-d4d8-4c53-86bc-a2cd11cb5f98 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:51.824065 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c828d4-c1d2-4c46-91c1-d00e92d3d1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:51.825159 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c828d4-c1d2-4c46-91c1-d00e92d3d1fe None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:51.825605 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c828d4-c1d2-4c46-91c1-d00e92d3d1fe None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:51.826042 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c828d4-c1d2-4c46-91c1-d00e92d3d1fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:51.839266 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925975f9-f96b-43ac-9299-94763b631b05 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:51.839577 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925975f9-f96b-43ac-9299-94763b631b05 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:51.839833 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925975f9-f96b-43ac-9299-94763b631b05 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:51.840712 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c828d4-c1d2-4c46-91c1-d00e92d3d1fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:51.841468 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 235/458] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:44:51.842370 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c828d4-c1d2-4c46-91c1-d00e92d3d1fe None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:51.847054 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2e2b5-2f63-403b-9ddc-5b79958c0d58 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:51.847506 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2e2b5-2f63-403b-9ddc-5b79958c0d58 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:51.847929 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2e2b5-2f63-403b-9ddc-5b79958c0d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:51.867775 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-b7c828d4-c1d2-4c46-91c1-d00e92d3d1fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:44:51.867775 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:51.867775 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:51.867775 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:51.867775 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:51.867775 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:51.867775 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:51.867775 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:51.867775 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:51.867775 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:51.867775 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:51.867775 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:51.867775 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:44:51.867775 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:44:51.867775 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:44:51.868890 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:44:51.868890 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:51.868890 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:51.868890 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:51.868890 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:51.868890 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:51.868890 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:51.868890 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:51.868890 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:44:51.868890 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:51.868890 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:51.868890 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:51.868890 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:51.868890 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:51.868890 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:51.868890 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:51.868890 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:51.869830 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:44:51.869830 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:44:51.869830 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:44:51.869830 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:44:51.869830 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 224/459] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:44:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:51.875328 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c828d4-c1d2-4c46-91c1-d00e92d3d1fe None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:51.890590 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-5ef2e2b5-2f63-403b-9ddc-5b79958c0d58 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:51.892004 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-5ef2e2b5-2f63-403b-9ddc-5b79958c0d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:51.961495 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 236/460] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 06 14:44:51.963053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea236d-88a3-4c09-bd44-4c2e68936d2e None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:51.963526 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea236d-88a3-4c09-bd44-4c2e68936d2e None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:51.963978 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea236d-88a3-4c09-bd44-4c2e68936d2e None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:51.965715 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 237/461] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:51.971692 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf3e9f-1ad6-4c32-ae2c-42fee5829f29 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:51.972455 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf3e9f-1ad6-4c32-ae2c-42fee5829f29 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:51.973002 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf3e9f-1ad6-4c32-ae2c-42fee5829f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:51.991280 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c828d4-c1d2-4c46-91c1-d00e92d3d1fe None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:52.000940 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-37bf3e9f-1ad6-4c32-ae2c-42fee5829f29 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:52.002378 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-37bf3e9f-1ad6-4c32-ae2c-42fee5829f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:52.033061 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877f4a4-19f0-467b-a5d1-bb4679e95cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:52.033784 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877f4a4-19f0-467b-a5d1-bb4679e95cfd None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:52.034087 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877f4a4-19f0-467b-a5d1-bb4679e95cfd None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:52.034489 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877f4a4-19f0-467b-a5d1-bb4679e95cfd None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:52.035226 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2877f4a4-19f0-467b-a5d1-bb4679e95cfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:52.035538 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2877f4a4-19f0-467b-a5d1-bb4679e95cfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:52.037826 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2877f4a4-19f0-467b-a5d1-bb4679e95cfd None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:52.052117 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 225/462] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:44:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:44:52.057529 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877f4a4-19f0-467b-a5d1-bb4679e95cfd None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:52.080882 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 238/463] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 06 14:44:52.088629 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bf3e9f-1ad6-4c32-ae2c-42fee5829f29 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:52.179694 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877f4a4-19f0-467b-a5d1-bb4679e95cfd None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:52.214308 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e09cf-9277-4c9c-9257-72b4d9934a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:52.215447 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e09cf-9277-4c9c-9257-72b4d9934a08 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:52.215922 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e09cf-9277-4c9c-9257-72b4d9934a08 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:52.216406 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e09cf-9277-4c9c-9257-72b4d9934a08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:52.228094 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e09cf-9277-4c9c-9257-72b4d9934a08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:52.229379 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bf3e9f-1ad6-4c32-ae2c-42fee5829f29 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:52.230115 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e09cf-9277-4c9c-9257-72b4d9934a08 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:52.248809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-758e09cf-9277-4c9c-9257-72b4d9934a08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:44:52.248809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:52.248809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:52.248809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:52.248809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:52.248809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:52.248809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:52.248809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:52.248809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:52.248809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:52.248809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:52.248809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:52.248809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:44:52.248809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:44:52.248809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:44:52.249679 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:44:52.249679 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:52.249679 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:52.249679 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:52.249679 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:52.249679 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:52.249679 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:52.249679 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:52.249679 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:44:52.249679 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:52.249679 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:52.249679 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:52.249679 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:52.249679 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:52.249679 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:52.249679 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:52.249679 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:52.250507 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:44:52.250507 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:44:52.250507 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:44:52.250507 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:44:52.252697 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 226/464] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:44:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:52.253734 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e09cf-9277-4c9c-9257-72b4d9934a08 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:52.266942 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2333f7fb-9e0c-48f0-bb14-6f19252c59b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:52.267934 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333f7fb-9e0c-48f0-bb14-6f19252c59b5 None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:52.268237 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333f7fb-9e0c-48f0-bb14-6f19252c59b5 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:52.268520 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333f7fb-9e0c-48f0-bb14-6f19252c59b5 None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:52.269560 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2333f7fb-9e0c-48f0-bb14-6f19252c59b5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:52.271263 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2333f7fb-9e0c-48f0-bb14-6f19252c59b5 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:52.296552 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 239/465] 10.222.0.23 () {60 vars in 1132 bytes} [Fri May 6 14:44:52 2022] POST /identity/v3/users => generated 246 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:44:52.297841 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2333f7fb-9e0c-48f0-bb14-6f19252c59b5 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:52.380573 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e09cf-9277-4c9c-9257-72b4d9934a08 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:52.416205 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2333f7fb-9e0c-48f0-bb14-6f19252c59b5 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:52.421573 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d544e5-21a3-46d4-a168-f629bc85e0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:52.422388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d544e5-21a3-46d4-a168-f629bc85e0f6 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:52.422673 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d544e5-21a3-46d4-a168-f629bc85e0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:52.422952 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d544e5-21a3-46d4-a168-f629bc85e0f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:52.434957 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d544e5-21a3-46d4-a168-f629bc85e0f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:52.436501 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d544e5-21a3-46d4-a168-f629bc85e0f6 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:52.452443 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48233551-af74-4a4d-8da9-b67be44abab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:52.453202 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48233551-af74-4a4d-8da9-b67be44abab9 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:52.453490 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48233551-af74-4a4d-8da9-b67be44abab9 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:52.453769 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48233551-af74-4a4d-8da9-b67be44abab9 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:52.454493 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48233551-af74-4a4d-8da9-b67be44abab9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:52.454793 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48233551-af74-4a4d-8da9-b67be44abab9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:52.455719 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-b3d544e5-21a3-46d4-a168-f629bc85e0f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:44:52.455719 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:52.455719 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:52.455719 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:52.455719 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:52.455719 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:52.455719 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:52.455719 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:52.455719 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:52.455719 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:52.455719 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:52.455719 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:52.455719 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:44:52.455719 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:44:52.455719 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:44:52.456557 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:44:52.456557 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:52.456557 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:52.456557 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:52.456557 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:52.456557 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:52.456557 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:52.456557 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:52.456557 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:44:52.456557 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:52.456557 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:52.456557 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:52.456557 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:52.456557 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:52.456557 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:52.456557 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:52.456557 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:52.457420 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:44:52.457420 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:44:52.457420 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:44:52.457420 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:44:52.457623 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48233551-af74-4a4d-8da9-b67be44abab9 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:52.457688 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 227/466] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:44:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:52.460141 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a485ffa-96f9-4ace-8820-41b8d3bce409 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:52.460141 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a485ffa-96f9-4ace-8820-41b8d3bce409 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:52.460141 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a485ffa-96f9-4ace-8820-41b8d3bce409 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:52.461051 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 228/467] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:52.465415 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a485ffa-96f9-4ace-8820-41b8d3bce409 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:52.479159 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 240/468] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:44:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:52.480733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e07a4-75ad-40a8-a585-ebadf945108f None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:52.481621 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e07a4-75ad-40a8-a585-ebadf945108f None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:52.482045 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e07a4-75ad-40a8-a585-ebadf945108f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:52.500847 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-891e07a4-75ad-40a8-a585-ebadf945108f None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:52.502288 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-891e07a4-75ad-40a8-a585-ebadf945108f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:52.575225 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 241/469] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:44:52.576344 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e07a4-75ad-40a8-a585-ebadf945108f None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:52.583960 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a485ffa-96f9-4ace-8820-41b8d3bce409 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:52.620884 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a4a73-1c3d-4264-8a77-91ddc593ad4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:52.621445 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a4a73-1c3d-4264-8a77-91ddc593ad4b None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:52.622307 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a4a73-1c3d-4264-8a77-91ddc593ad4b None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:52.622307 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a4a73-1c3d-4264-8a77-91ddc593ad4b None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:52.622710 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9a4a73-1c3d-4264-8a77-91ddc593ad4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:52.623015 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9a4a73-1c3d-4264-8a77-91ddc593ad4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:52.624387 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9a4a73-1c3d-4264-8a77-91ddc593ad4b None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:52.636152 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 229/470] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:44:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:52.637391 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d10def-f6df-4a27-a7ca-e94a9a933570 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:52.637682 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d10def-f6df-4a27-a7ca-e94a9a933570 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:52.637960 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d10def-f6df-4a27-a7ca-e94a9a933570 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:52.639464 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 230/471] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:52.640167 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d10def-f6df-4a27-a7ca-e94a9a933570 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:52.689892 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e07a4-75ad-40a8-a585-ebadf945108f None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:52.725796 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42f0cc0-2fa9-44cd-9e64-658e15ae4e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:52.726573 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f0cc0-2fa9-44cd-9e64-658e15ae4e0c None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:52.727222 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f0cc0-2fa9-44cd-9e64-658e15ae4e0c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:52.727512 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f0cc0-2fa9-44cd-9e64-658e15ae4e0c None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:52.728723 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42f0cc0-2fa9-44cd-9e64-658e15ae4e0c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:52.730638 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42f0cc0-2fa9-44cd-9e64-658e15ae4e0c None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:52.745123 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d10def-f6df-4a27-a7ca-e94a9a933570 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:52.763108 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 242/472] 10.222.0.23 () {60 vars in 1132 bytes} [Fri May 6 14:44:52 2022] POST /identity/v3/users => generated 248 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:44:52.763108 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42f0cc0-2fa9-44cd-9e64-658e15ae4e0c None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:52.780936 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0817e99d-1d45-4f36-a773-e6f1911eced5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:52.781758 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0817e99d-1d45-4f36-a773-e6f1911eced5 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:52.782035 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0817e99d-1d45-4f36-a773-e6f1911eced5 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:52.782304 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0817e99d-1d45-4f36-a773-e6f1911eced5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:52.794901 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0817e99d-1d45-4f36-a773-e6f1911eced5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:52.796502 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0817e99d-1d45-4f36-a773-e6f1911eced5 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:52.818456 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-0817e99d-1d45-4f36-a773-e6f1911eced5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:44:52.818456 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:52.818456 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:52.818456 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:52.818456 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:52.818456 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:52.818456 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:52.818456 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:52.818456 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:52.818456 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:52.818456 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:52.818456 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:52.818456 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:44:52.818456 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:44:52.818456 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:44:52.820135 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:44:52.820135 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:52.820135 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:52.820135 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:52.820135 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:52.820135 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:52.820135 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:52.820135 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:52.820135 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:44:52.820135 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:52.820135 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:52.820135 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:52.820135 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:52.820135 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:52.820135 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:52.820135 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:52.820135 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:52.821132 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:44:52.821132 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:44:52.821132 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:44:52.821132 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:44:52.821132 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 231/473] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:44:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:52.822420 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3e9aa-3ab8-4f5b-bff8-072504bdb498 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:52.822776 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3e9aa-3ab8-4f5b-bff8-072504bdb498 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:52.823122 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3e9aa-3ab8-4f5b-bff8-072504bdb498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:52.842337 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-c1e3e9aa-3ab8-4f5b-bff8-072504bdb498 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:52.843612 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-c1e3e9aa-3ab8-4f5b-bff8-072504bdb498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:52.907510 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42f0cc0-2fa9-44cd-9e64-658e15ae4e0c None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:52.927700 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 232/474] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:44:52.929217 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3e9aa-3ab8-4f5b-bff8-072504bdb498 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:52.951266 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2096169-8649-45c2-86f7-7fdd5703750d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:52.951957 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2096169-8649-45c2-86f7-7fdd5703750d None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:52.952434 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2096169-8649-45c2-86f7-7fdd5703750d None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:52.952563 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2096169-8649-45c2-86f7-7fdd5703750d None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:52.953287 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2096169-8649-45c2-86f7-7fdd5703750d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:52.954724 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2096169-8649-45c2-86f7-7fdd5703750d None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:52.981652 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 243/475] 10.222.0.23 () {60 vars in 1132 bytes} [Fri May 6 14:44:52 2022] POST /identity/v3/users => generated 248 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:44:52.982749 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2096169-8649-45c2-86f7-7fdd5703750d None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:53.065741 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3e9aa-3ab8-4f5b-bff8-072504bdb498 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:53.101211 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2096169-8649-45c2-86f7-7fdd5703750d None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:53.111547 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc756b1-f586-4058-8fbc-547012f66630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:53.112703 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc756b1-f586-4058-8fbc-547012f66630 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:53.112993 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc756b1-f586-4058-8fbc-547012f66630 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:53.113266 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc756b1-f586-4058-8fbc-547012f66630 None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:53.114361 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc756b1-f586-4058-8fbc-547012f66630 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:53.114672 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc756b1-f586-4058-8fbc-547012f66630 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:53.116462 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc756b1-f586-4058-8fbc-547012f66630 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:53.134360 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 233/476] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:44:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:53.141736 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc756b1-f586-4058-8fbc-547012f66630 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:53.150617 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b2bc1-f0b7-4ab9-b1a8-75387dd3539a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:53.151308 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b2bc1-f0b7-4ab9-b1a8-75387dd3539a None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:53.151590 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b2bc1-f0b7-4ab9-b1a8-75387dd3539a None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:53.151862 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b2bc1-f0b7-4ab9-b1a8-75387dd3539a None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:53.152630 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083b2bc1-f0b7-4ab9-b1a8-75387dd3539a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:53.154161 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083b2bc1-f0b7-4ab9-b1a8-75387dd3539a None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:53.162437 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.sql.core [None req-083b2bc1-f0b7-4ab9-b1a8-75387dd3539a None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 06 14:44:53.162437 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 06 14:44:53.162437 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [parameters: {'id': '991c6e52a07a4f029ff37c8c32663fe2', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 06 14:44:53.162437 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=79888) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 06 14:44:53.179110 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-083b2bc1-f0b7-4ab9-b1a8-75387dd3539a None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 06 14:44:53.179110 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:53.179110 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 06 14:44:53.179110 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self.dialect.do_execute( May 06 14:44:53.179110 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 06 14:44:53.179110 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 06 14:44:53.179110 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 06 14:44:53.179110 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application result = self._query(query) May 06 14:44:53.179110 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 06 14:44:53.179110 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application conn.query(q) May 06 14:44:53.179110 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 06 14:44:53.179110 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 06 14:44:53.179110 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 06 14:44:53.179110 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application result.read() May 06 14:44:53.179110 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 06 14:44:53.180057 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 06 14:44:53.180057 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 06 14:44:53.180057 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application packet.raise_for_error() May 06 14:44:53.180057 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 06 14:44:53.180057 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 06 14:44:53.180057 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 06 14:44:53.180057 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 06 14:44:53.180057 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 06 14:44:53.180057 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:44:53.180057 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 06 14:44:53.180057 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:44:53.180057 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:53.180057 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 06 14:44:53.180057 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 06 14:44:53.180057 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 06 14:44:53.180057 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return ref.to_dict() May 06 14:44:53.180057 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 06 14:44:53.180057 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application next(self.gen) May 06 14:44:53.181012 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 06 14:44:53.181012 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application yield resource May 06 14:44:53.181012 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 06 14:44:53.181012 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application next(self.gen) May 06 14:44:53.181012 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 06 14:44:53.181012 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self.session.rollback() May 06 14:44:53.181012 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 06 14:44:53.181012 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self.force_reraise() May 06 14:44:53.181012 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 06 14:44:53.181012 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise self.value May 06 14:44:53.181012 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 06 14:44:53.181012 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 06 14:44:53.181012 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 06 14:44:53.181012 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application session.commit() May 06 14:44:53.181012 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 06 14:44:53.181012 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 06 14:44:53.181012 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 06 14:44:53.181857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._prepare_impl() May 06 14:44:53.181857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 06 14:44:53.181857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self.session.flush() May 06 14:44:53.181857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 06 14:44:53.181857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._flush(objects) May 06 14:44:53.181857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 06 14:44:53.181857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 06 14:44:53.181857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 06 14:44:53.181857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application compat.raise_( May 06 14:44:53.181857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 06 14:44:53.181857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception May 06 14:44:53.181857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 06 14:44:53.181857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application flush_context.execute() May 06 14:44:53.181857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 06 14:44:53.181857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rec.execute(self) May 06 14:44:53.181857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 06 14:44:53.181857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 06 14:44:53.182809 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 06 14:44:53.182809 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application _emit_insert_statements( May 06 14:44:53.182809 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 06 14:44:53.182809 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application c = connection._execute_20( May 06 14:44:53.182809 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 06 14:44:53.182809 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 06 14:44:53.182809 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 06 14:44:53.182809 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 06 14:44:53.182809 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 06 14:44:53.182809 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ret = self._execute_context( May 06 14:44:53.182809 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 06 14:44:53.182809 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 06 14:44:53.182809 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 06 14:44:53.182809 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 06 14:44:53.182809 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 06 14:44:53.182809 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception May 06 14:44:53.183629 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 06 14:44:53.183629 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self.dialect.do_execute( May 06 14:44:53.183629 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 06 14:44:53.183629 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 06 14:44:53.183629 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 06 14:44:53.183629 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application result = self._query(query) May 06 14:44:53.183629 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 06 14:44:53.183629 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application conn.query(q) May 06 14:44:53.183629 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 06 14:44:53.183629 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 06 14:44:53.183629 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 06 14:44:53.183629 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application result.read() May 06 14:44:53.183629 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 06 14:44:53.183629 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 06 14:44:53.183629 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 06 14:44:53.183629 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application packet.raise_for_error() May 06 14:44:53.185648 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 06 14:44:53.185648 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 06 14:44:53.185648 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 06 14:44:53.185648 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 06 14:44:53.185648 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 06 14:44:53.185648 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 06 14:44:53.185648 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [parameters: {'id': '991c6e52a07a4f029ff37c8c32663fe2', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 06 14:44:53.185648 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 06 14:44:53.185648 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:44:53.185648 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 06 14:44:53.185648 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:44:53.185648 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:53.185648 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:53.185648 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:53.185648 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:53.185648 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:53.186690 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:53.186690 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:53.186690 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:53.186690 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:53.186690 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:53.186690 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:53.186690 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 06 14:44:53.186690 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 06 14:44:53.186690 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:53.186690 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:53.186690 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 06 14:44:53.186690 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 06 14:44:53.186690 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 06 14:44:53.186690 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 06 14:44:53.186690 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 06 14:44:53.186690 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:44:53.187713 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 244/477] 10.222.0.23 () {60 vars in 1131 bytes} [Fri May 6 14:44:52 2022] POST /identity/v3/roles => generated 144 bytes in 198 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:53.195249 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b2bc1-f0b7-4ab9-b1a8-75387dd3539a None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:53.291693 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc756b1-f586-4058-8fbc-547012f66630 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:53.312823 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b2bc1-f0b7-4ab9-b1a8-75387dd3539a None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:53.338954 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3896b3f-054f-4381-8ae4-2f4f49312ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:53.339920 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3896b3f-054f-4381-8ae4-2f4f49312ec3 None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:53.340387 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3896b3f-054f-4381-8ae4-2f4f49312ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:53.340868 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3896b3f-054f-4381-8ae4-2f4f49312ec3 None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:53.341752 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3896b3f-054f-4381-8ae4-2f4f49312ec3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:53.343227 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3896b3f-054f-4381-8ae4-2f4f49312ec3 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:53.367521 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 234/478] 10.222.0.23 () {60 vars in 1132 bytes} [Fri May 6 14:44:53 2022] POST /identity/v3/users => generated 249 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:44:53.374149 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4b6dc-bf31-49a0-99d8-64ed503aa29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:53.374589 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4b6dc-bf31-49a0-99d8-64ed503aa29f None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:53.374940 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4b6dc-bf31-49a0-99d8-64ed503aa29f None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:53.375222 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4b6dc-bf31-49a0-99d8-64ed503aa29f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:53.388738 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e4b6dc-bf31-49a0-99d8-64ed503aa29f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:53.397755 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-70e4b6dc-bf31-49a0-99d8-64ed503aa29f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 06 14:44:53.397755 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:53.397755 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:53.397755 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:53.397755 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:53.397755 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:53.397755 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:53.397755 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:53.397755 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:53.397755 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:53.397755 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:53.397755 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:53.397755 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:53.397755 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:53.397755 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:53.397755 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:53.398760 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:53.398760 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:53.398760 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:53.398760 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:53.398760 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:53.398760 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:53.398760 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:53.398760 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:53.398760 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:53.398760 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:53.398760 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:53.398760 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:44:53.398760 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:53.398760 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:53.398760 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:53.398760 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:53.399602 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:53.399602 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:53.399602 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:53.399602 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:53.399602 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:53.399602 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:53.399602 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:53.399602 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:53.399602 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:53.399602 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:53.399602 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 06 14:44:53.399602 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:44:53.399602 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 245/479] 10.222.0.23 () {56 vars in 1114 bytes} [Fri May 6 14:44:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:44:53.405165 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3896b3f-054f-4381-8ae4-2f4f49312ec3 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:53.522965 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3896b3f-054f-4381-8ae4-2f4f49312ec3 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:53.559735 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a54a6c-965c-4535-a331-9fbf75642e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:53.560413 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a54a6c-965c-4535-a331-9fbf75642e59 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:53.560726 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a54a6c-965c-4535-a331-9fbf75642e59 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:53.561009 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a54a6c-965c-4535-a331-9fbf75642e59 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:53.561768 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a54a6c-965c-4535-a331-9fbf75642e59 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:53.562082 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a54a6c-965c-4535-a331-9fbf75642e59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:53.563533 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a54a6c-965c-4535-a331-9fbf75642e59 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:53.573269 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 235/480] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:44:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:55.023372 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea159e-dfb4-4f9e-9c5b-cff201e3b15d None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:55.023893 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea159e-dfb4-4f9e-9c5b-cff201e3b15d None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:55.023997 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea159e-dfb4-4f9e-9c5b-cff201e3b15d None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:55.025591 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 246/481] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:55.031515 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d8d06-53f0-474b-b071-82d829a301ee None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:55.031938 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d8d06-53f0-474b-b071-82d829a301ee None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:55.032112 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d8d06-53f0-474b-b071-82d829a301ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:55.049312 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-a56d8d06-53f0-474b-b071-82d829a301ee None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:55.050550 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-a56d8d06-53f0-474b-b071-82d829a301ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:55.129279 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 236/482] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:44:55.186207 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea159e-dfb4-4f9e-9c5b-cff201e3b15d None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:55.324031 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea159e-dfb4-4f9e-9c5b-cff201e3b15d None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:55.364839 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c93a9-d5c5-452a-b7aa-f6f493282853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:55.365702 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c93a9-d5c5-452a-b7aa-f6f493282853 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:55.366500 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c93a9-d5c5-452a-b7aa-f6f493282853 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:55.366785 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c93a9-d5c5-452a-b7aa-f6f493282853 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:55.385460 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98c93a9-d5c5-452a-b7aa-f6f493282853 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:55.393170 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-e98c93a9-d5c5-452a-b7aa-f6f493282853 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 06 14:44:55.393170 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:55.393170 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:55.393170 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:55.393170 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:55.393170 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:55.393170 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:55.393170 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:55.393170 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:55.393170 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:55.393170 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:55.393170 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:55.393170 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:55.393170 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:55.393170 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:55.393170 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:55.394184 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:55.394184 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:55.394184 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:55.394184 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:55.394184 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:55.394184 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:55.394184 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:55.394184 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:55.394184 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:55.394184 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:55.394184 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:55.394184 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:44:55.394184 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:55.394184 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:55.394184 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:55.394184 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:55.395360 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:55.395360 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:55.395360 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:55.395360 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:55.395360 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:55.395360 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:55.395360 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:55.395360 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:55.395360 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:55.395360 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:55.395360 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 06 14:44:55.395360 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:44:55.395360 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 247/483] 10.222.0.23 () {56 vars in 1102 bytes} [Fri May 6 14:44:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:55.402122 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d8d06-53f0-474b-b071-82d829a301ee None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:55.534279 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d8d06-53f0-474b-b071-82d829a301ee None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:55.576174 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989bb003-0cb2-490f-8070-8da4a2a4c423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:55.576800 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989bb003-0cb2-490f-8070-8da4a2a4c423 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:55.577088 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989bb003-0cb2-490f-8070-8da4a2a4c423 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:55.577364 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989bb003-0cb2-490f-8070-8da4a2a4c423 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:55.578116 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989bb003-0cb2-490f-8070-8da4a2a4c423 None admin] RBAC: Adding query filter params (name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:55.578423 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989bb003-0cb2-490f-8070-8da4a2a4c423 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:55.579790 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989bb003-0cb2-490f-8070-8da4a2a4c423 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:55.589032 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 237/484] 10.222.0.23 () {56 vars in 1103 bytes} [Fri May 6 14:44:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:55.594794 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c93a9-d5c5-452a-b7aa-f6f493282853 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:55.652911 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a3b60a-054e-4f3b-b291-e55446efabd2 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:55.653327 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a3b60a-054e-4f3b-b291-e55446efabd2 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:55.653484 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a3b60a-054e-4f3b-b291-e55446efabd2 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:55.654806 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 238/485] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:44:55.660018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b3a11-578b-4314-b842-49dedfca00a5 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:55.660303 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b3a11-578b-4314-b842-49dedfca00a5 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:55.660576 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b3a11-578b-4314-b842-49dedfca00a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:55.680922 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-d44b3a11-578b-4314-b842-49dedfca00a5 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:55.682791 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d44b3a11-578b-4314-b842-49dedfca00a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:55.764550 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c93a9-d5c5-452a-b7aa-f6f493282853 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:55.812189 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 239/486] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 06 14:44:55.821704 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78168568-1664-42b9-bf58-4494f90a56e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:55.822528 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78168568-1664-42b9-bf58-4494f90a56e3 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:55.822812 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78168568-1664-42b9-bf58-4494f90a56e3 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:55.823081 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78168568-1664-42b9-bf58-4494f90a56e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:55.835349 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78168568-1664-42b9-bf58-4494f90a56e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:55.836651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78168568-1664-42b9-bf58-4494f90a56e3 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:55.849182 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44b3a11-578b-4314-b842-49dedfca00a5 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:55.870086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-78168568-1664-42b9-bf58-4494f90a56e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:44:55.870086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:55.870086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:55.870086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:55.870086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:55.870086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:55.870086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:55.870086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:55.870086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:55.870086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:55.870086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:55.870086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:55.870086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:44:55.870086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:44:55.870086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:44:55.872114 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:44:55.872114 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:55.872114 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:55.872114 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:55.872114 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:55.872114 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:55.872114 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:55.872114 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:55.872114 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:44:55.872114 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:55.872114 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:55.872114 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:55.872114 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:55.872114 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:55.872114 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:55.872114 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:55.872114 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:55.874261 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:44:55.874261 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:44:55.874261 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:44:55.874261 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:44:55.874261 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 248/487] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:44:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:55.881082 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78168568-1664-42b9-bf58-4494f90a56e3 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:56.035644 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44b3a11-578b-4314-b842-49dedfca00a5 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:56.050746 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78168568-1664-42b9-bf58-4494f90a56e3 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:56.071051 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c7e331-8bc8-4942-8818-f849c1be11e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:56.072120 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7e331-8bc8-4942-8818-f849c1be11e9 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:56.072545 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7e331-8bc8-4942-8818-f849c1be11e9 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:56.072990 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7e331-8bc8-4942-8818-f849c1be11e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:56.083946 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c7e331-8bc8-4942-8818-f849c1be11e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:56.084990 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688df53-9420-4e89-8a98-6954ead38b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:56.085712 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688df53-9420-4e89-8a98-6954ead38b90 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:56.085944 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688df53-9420-4e89-8a98-6954ead38b90 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:56.086225 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688df53-9420-4e89-8a98-6954ead38b90 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:56.086948 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2688df53-9420-4e89-8a98-6954ead38b90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:56.087256 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2688df53-9420-4e89-8a98-6954ead38b90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:56.088635 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2688df53-9420-4e89-8a98-6954ead38b90 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:56.093414 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-51c7e331-8bc8-4942-8818-f849c1be11e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 06 14:44:56.093414 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:56.093414 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:56.093414 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:56.093414 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:56.093414 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:56.093414 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:56.093414 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:56.093414 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:56.093414 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:56.093414 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:56.093414 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:56.093414 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:56.093414 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:56.093414 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:56.093414 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:56.094491 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:56.094491 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:56.094491 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:56.094491 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:56.094491 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:56.094491 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:56.094491 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:56.094491 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:56.094491 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:56.094491 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:56.094491 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:56.094491 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:44:56.094491 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:56.094491 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:56.094491 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:56.094491 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:56.095481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:56.095481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:56.095481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:56.095481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:56.095481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:56.095481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:56.095481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:56.095481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:56.095481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:56.095481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:56.095481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 06 14:44:56.095481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:44:56.097683 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 240/488] 10.222.0.23 () {56 vars in 1102 bytes} [Fri May 6 14:44:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:44:56.099143 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f02c2-0413-4d04-be5f-2c591d01935a None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:56.099585 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f02c2-0413-4d04-be5f-2c591d01935a None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:56.100543 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f02c2-0413-4d04-be5f-2c591d01935a None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:56.101866 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 249/489] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:44:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:56.102109 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 241/490] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:56.107108 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004f02c2-0413-4d04-be5f-2c591d01935a None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:56.107239 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688df53-9420-4e89-8a98-6954ead38b90 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:56.214959 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004f02c2-0413-4d04-be5f-2c591d01935a None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:56.220079 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688df53-9420-4e89-8a98-6954ead38b90 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:56.250977 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c4a291-bd86-4701-91eb-845c4fb07719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:56.252147 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4a291-bd86-4701-91eb-845c4fb07719 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:56.252693 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4a291-bd86-4701-91eb-845c4fb07719 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:56.253185 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4a291-bd86-4701-91eb-845c4fb07719 None admin] PATH_INFO: `/v3/users/nova` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:56.262194 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162854ce-e283-4432-a180-1d10fbb03806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:56.262386 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c4a291-bd86-4701-91eb-845c4fb07719 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:56.263467 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162854ce-e283-4432-a180-1d10fbb03806 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:56.263882 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c4a291-bd86-4701-91eb-845c4fb07719 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:56.264155 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162854ce-e283-4432-a180-1d10fbb03806 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:56.264797 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162854ce-e283-4432-a180-1d10fbb03806 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:56.265716 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162854ce-e283-4432-a180-1d10fbb03806 None admin] RBAC: Adding query filter params (name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:56.266211 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162854ce-e283-4432-a180-1d10fbb03806 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:56.267733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162854ce-e283-4432-a180-1d10fbb03806 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:56.277214 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 250/491] 10.222.0.23 () {56 vars in 1103 bytes} [Fri May 6 14:44:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:56.278780 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c180ad-145b-4da4-8a03-7d29c4554cdd None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:56.279243 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c180ad-145b-4da4-8a03-7d29c4554cdd None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:56.279714 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c180ad-145b-4da4-8a03-7d29c4554cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:56.284553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-02c4a291-bd86-4701-91eb-845c4fb07719 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 06 14:44:56.284553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:56.284553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:56.284553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:56.284553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:56.284553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:56.284553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:56.284553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:56.284553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:56.284553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:56.284553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:56.284553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:56.284553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 06 14:44:56.284553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_user(user_id) May 06 14:44:56.284553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 06 14:44:56.284553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 06 14:44:56.285475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:56.285475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:56.285475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 06 14:44:56.285475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:44:56.285475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 06 14:44:56.285475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:44:56.285475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:56.285475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:56.285475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:56.285475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:56.285475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:56.285475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:44:56.285475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:56.285475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:56.285475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:56.285475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:56.285475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:56.285475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:56.286356 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:56.286356 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:56.286356 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 06 14:44:56.286356 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 06 14:44:56.286356 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 06 14:44:56.286356 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 06 14:44:56.286356 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 06 14:44:56.286356 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 06 14:44:56.286356 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 06 14:44:56.286356 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:44:56.286356 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 242/492] 10.222.0.23 () {56 vars in 1093 bytes} [Fri May 6 14:44:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:56.286938 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c4a291-bd86-4701-91eb-845c4fb07719 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:56.305949 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-a8c180ad-145b-4da4-8a03-7d29c4554cdd None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:56.307450 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-a8c180ad-145b-4da4-8a03-7d29c4554cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:56.376946 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 251/493] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:44:56.378175 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c180ad-145b-4da4-8a03-7d29c4554cdd None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:56.410866 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c4a291-bd86-4701-91eb-845c4fb07719 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:56.445576 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a310bb-bce1-4fae-9262-8ea501483bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:56.446372 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a310bb-bce1-4fae-9262-8ea501483bda None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:56.446712 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a310bb-bce1-4fae-9262-8ea501483bda None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:56.447000 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a310bb-bce1-4fae-9262-8ea501483bda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:56.458481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a310bb-bce1-4fae-9262-8ea501483bda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:56.460584 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a310bb-bce1-4fae-9262-8ea501483bda None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:56.478580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-f6a310bb-bce1-4fae-9262-8ea501483bda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:44:56.478580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:56.478580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:56.478580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:56.478580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:56.478580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:56.478580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:56.478580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:56.478580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:56.478580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:56.478580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:56.478580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:56.478580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:44:56.478580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:44:56.478580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:44:56.479641 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:44:56.479641 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:56.479641 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:56.479641 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:56.479641 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:56.479641 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:56.479641 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:56.479641 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:56.479641 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:44:56.479641 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:56.479641 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:56.479641 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:56.479641 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:56.479641 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:56.479641 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:56.479641 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:56.479641 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:56.480523 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:44:56.480523 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:44:56.480523 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:44:56.480523 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:44:56.480523 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 243/494] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:44:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:56.481183 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b7517-8d71-408d-a4f8-89bed425f112 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:56.481756 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b7517-8d71-408d-a4f8-89bed425f112 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:56.481756 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b7517-8d71-408d-a4f8-89bed425f112 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:56.483091 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 244/495] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:56.484724 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b7517-8d71-408d-a4f8-89bed425f112 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:56.489529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c180ad-145b-4da4-8a03-7d29c4554cdd None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:56.523991 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20396d1a-26f7-4efb-b7ba-adf0a8c83420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:56.524706 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20396d1a-26f7-4efb-b7ba-adf0a8c83420 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:56.524994 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20396d1a-26f7-4efb-b7ba-adf0a8c83420 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:56.525266 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20396d1a-26f7-4efb-b7ba-adf0a8c83420 None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:56.526290 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20396d1a-26f7-4efb-b7ba-adf0a8c83420 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:56.526595 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20396d1a-26f7-4efb-b7ba-adf0a8c83420 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:56.528024 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20396d1a-26f7-4efb-b7ba-adf0a8c83420 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:56.570866 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 252/496] 10.222.0.23 () {56 vars in 1133 bytes} [Fri May 6 14:44:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:56.571711 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20396d1a-26f7-4efb-b7ba-adf0a8c83420 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:56.598768 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b7517-8d71-408d-a4f8-89bed425f112 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:56.634187 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c4a549-bd5d-4f36-aa60-5e00ebd6123e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:56.634905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c4a549-bd5d-4f36-aa60-5e00ebd6123e None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:56.635184 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c4a549-bd5d-4f36-aa60-5e00ebd6123e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:56.635455 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c4a549-bd5d-4f36-aa60-5e00ebd6123e None admin] PATH_INFO: `/v3/roles/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:56.645098 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c4a549-bd5d-4f36-aa60-5e00ebd6123e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:56.653283 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-43c4a549-bd5d-4f36-aa60-5e00ebd6123e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 06 14:44:56.653283 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:56.653283 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:56.653283 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:56.653283 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:56.653283 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:56.653283 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:56.653283 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:56.653283 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:56.653283 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:56.653283 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:56.653283 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:56.653283 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:56.653283 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:56.653283 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:56.653283 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:56.654379 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:56.654379 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:56.654379 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:56.654379 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:56.654379 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:56.654379 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:56.654379 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:56.654379 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:56.654379 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:56.654379 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:56.654379 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:56.654379 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:44:56.654379 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:56.654379 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:56.654379 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:56.654379 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:56.655252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:56.655252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:56.655252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:56.655252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:56.655252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:56.655252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:56.655252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:56.655252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:56.655252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:56.655252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:56.655252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 06 14:44:56.655252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:44:56.655252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 245/497] 10.222.0.23 () {56 vars in 1102 bytes} [Fri May 6 14:44:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:56.656174 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b007eb1-c438-4b8c-829c-8b5d28e791f9 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:56.656441 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b007eb1-c438-4b8c-829c-8b5d28e791f9 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:56.656740 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b007eb1-c438-4b8c-829c-8b5d28e791f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:56.671413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20396d1a-26f7-4efb-b7ba-adf0a8c83420 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:56.672893 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-3b007eb1-c438-4b8c-829c-8b5d28e791f9 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:56.674085 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-3b007eb1-c438-4b8c-829c-8b5d28e791f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:56.703887 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb10119-4483-48e2-b2a9-7c8f7f7280f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:56.704539 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb10119-4483-48e2-b2a9-7c8f7f7280f2 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:56.704852 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb10119-4483-48e2-b2a9-7c8f7f7280f2 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:56.705126 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb10119-4483-48e2-b2a9-7c8f7f7280f2 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:56.705835 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb10119-4483-48e2-b2a9-7c8f7f7280f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:56.706137 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb10119-4483-48e2-b2a9-7c8f7f7280f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:56.707493 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb10119-4483-48e2-b2a9-7c8f7f7280f2 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:56.718314 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 253/498] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:44:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:56.719591 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554ed0a0-0916-4a11-8f86-01ca17443615 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:56.719890 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554ed0a0-0916-4a11-8f86-01ca17443615 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:56.720165 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554ed0a0-0916-4a11-8f86-01ca17443615 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:56.721395 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 254/499] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:44:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:44:56.723259 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554ed0a0-0916-4a11-8f86-01ca17443615 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:56.743186 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 246/500] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:44:56.744022 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b007eb1-c438-4b8c-829c-8b5d28e791f9 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:56.822711 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554ed0a0-0916-4a11-8f86-01ca17443615 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:56.836532 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b007eb1-c438-4b8c-829c-8b5d28e791f9 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:56.854909 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ad682-9c6d-4e64-8cc1-47f429bc6f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:56.855558 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ad682-9c6d-4e64-8cc1-47f429bc6f42 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:56.855841 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ad682-9c6d-4e64-8cc1-47f429bc6f42 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:56.856121 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ad682-9c6d-4e64-8cc1-47f429bc6f42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:56.869169 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6cd78f-318a-4169-b2d7-4f57fae54680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:56.869828 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6cd78f-318a-4169-b2d7-4f57fae54680 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:56.870108 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6cd78f-318a-4169-b2d7-4f57fae54680 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:56.870378 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6cd78f-318a-4169-b2d7-4f57fae54680 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:56.871107 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6cd78f-318a-4169-b2d7-4f57fae54680 None admin] RBAC: Adding query filter params (name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:56.871409 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6cd78f-318a-4169-b2d7-4f57fae54680 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:56.872789 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6cd78f-318a-4169-b2d7-4f57fae54680 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:56.880616 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 247/501] 10.222.0.23 () {56 vars in 1103 bytes} [Fri May 6 14:44:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:56.881439 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6cd78f-318a-4169-b2d7-4f57fae54680 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:56.921467 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283ad682-9c6d-4e64-8cc1-47f429bc6f42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:56.922861 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283ad682-9c6d-4e64-8cc1-47f429bc6f42 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:56.973352 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6cd78f-318a-4169-b2d7-4f57fae54680 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:57.005369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a5682-4cd8-4caf-a9e7-ba6ead338b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:57.006211 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a5682-4cd8-4caf-a9e7-ba6ead338b7c None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:57.006652 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a5682-4cd8-4caf-a9e7-ba6ead338b7c None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:57.006931 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a5682-4cd8-4caf-a9e7-ba6ead338b7c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:57.014825 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8a5682-4cd8-4caf-a9e7-ba6ead338b7c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:57.015106 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 255/502] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:44:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 06 14:44:57.016342 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8a5682-4cd8-4caf-a9e7-ba6ead338b7c None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:57.016448 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c343e59-42d6-4e53-9b28-0ef27d3e7b35 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:57.016653 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c343e59-42d6-4e53-9b28-0ef27d3e7b35 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:57.016926 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c343e59-42d6-4e53-9b28-0ef27d3e7b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:57.034225 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-2c8a5682-4cd8-4caf-a9e7-ba6ead338b7c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 06 14:44:57.034225 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:57.034225 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:57.034225 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:57.034225 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:57.034225 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:57.034225 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:57.034225 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:57.034225 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:57.034225 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:57.034225 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:57.034225 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:57.034225 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 06 14:44:57.034225 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_user(user_id) May 06 14:44:57.034225 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 06 14:44:57.034225 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 06 14:44:57.035139 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:57.035139 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:57.035139 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 06 14:44:57.035139 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:44:57.035139 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 06 14:44:57.035139 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:44:57.035139 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:57.035139 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:57.035139 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:57.035139 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:57.035139 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:57.035139 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:44:57.035139 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:57.035139 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:57.035139 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:57.035139 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:57.035139 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:57.035139 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:57.036481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:57.036481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:57.036481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 06 14:44:57.036481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 06 14:44:57.036481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 06 14:44:57.036481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 06 14:44:57.036481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 06 14:44:57.036481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 06 14:44:57.036481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 06 14:44:57.036481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:44:57.038175 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-0c343e59-42d6-4e53-9b28-0ef27d3e7b35 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:44:57.038248 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 248/503] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:44:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:57.038320 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-0c343e59-42d6-4e53-9b28-0ef27d3e7b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:44:57.038387 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a5682-4cd8-4caf-a9e7-ba6ead338b7c None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:57.099391 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 256/504] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:44:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:44:57.100863 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c343e59-42d6-4e53-9b28-0ef27d3e7b35 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:57.132541 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a5682-4cd8-4caf-a9e7-ba6ead338b7c None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:57.166466 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6adad04-3e8d-4674-913a-de2ed73fedae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:57.167371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6adad04-3e8d-4674-913a-de2ed73fedae None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:57.167675 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6adad04-3e8d-4674-913a-de2ed73fedae None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:57.167955 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6adad04-3e8d-4674-913a-de2ed73fedae None admin] PATH_INFO: `/v3/roles/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:57.177813 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6adad04-3e8d-4674-913a-de2ed73fedae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:44:57.185467 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-a6adad04-3e8d-4674-913a-de2ed73fedae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 06 14:44:57.185467 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:57.185467 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:57.185467 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:57.185467 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:57.185467 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:57.185467 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:57.185467 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:57.185467 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:57.185467 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:57.185467 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:57.185467 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:57.185467 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:44:57.185467 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:44:57.185467 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:44:57.185467 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:44:57.186400 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:44:57.186400 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:44:57.186400 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:44:57.186400 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:44:57.186400 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:57.186400 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:57.186400 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:57.186400 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:57.186400 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:57.186400 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:57.186400 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:57.186400 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:44:57.186400 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:57.186400 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:57.186400 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:57.186400 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:57.187207 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:57.187207 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:57.187207 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:57.187207 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:57.187207 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:44:57.187207 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:44:57.187207 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:44:57.187207 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:44:57.187207 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:44:57.187207 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:44:57.187207 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 06 14:44:57.187207 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:44:57.187207 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 249/505] 10.222.0.23 () {56 vars in 1102 bytes} [Fri May 6 14:44:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:57.187861 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6adad04-3e8d-4674-913a-de2ed73fedae None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:57.207263 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c343e59-42d6-4e53-9b28-0ef27d3e7b35 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:57.241043 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60517598-f8f1-47fc-9a2b-b5996771769b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:57.241829 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60517598-f8f1-47fc-9a2b-b5996771769b None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:57.242116 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60517598-f8f1-47fc-9a2b-b5996771769b None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:57.242395 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60517598-f8f1-47fc-9a2b-b5996771769b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:57.251914 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60517598-f8f1-47fc-9a2b-b5996771769b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:57.253388 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60517598-f8f1-47fc-9a2b-b5996771769b None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:57.270198 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-60517598-f8f1-47fc-9a2b-b5996771769b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:44:57.270198 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:57.270198 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:57.270198 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:57.270198 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:57.270198 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:57.270198 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:57.270198 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:57.270198 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:57.270198 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:57.270198 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:57.270198 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:57.270198 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:44:57.270198 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:44:57.270198 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:44:57.271149 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:44:57.271149 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:57.271149 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:57.271149 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:57.271149 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:57.271149 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:57.271149 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:57.271149 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:57.271149 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:44:57.271149 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:57.271149 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:57.271149 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:57.271149 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:57.271149 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:57.271149 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:57.271149 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:57.271149 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:57.272555 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:44:57.272555 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:44:57.272555 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:44:57.272555 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:44:57.272555 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 257/506] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:44:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:57.272872 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60517598-f8f1-47fc-9a2b-b5996771769b None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:57.283987 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6adad04-3e8d-4674-913a-de2ed73fedae None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:57.315383 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d434020-1cf9-4948-a9ae-fde7d56330c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:57.316161 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d434020-1cf9-4948-a9ae-fde7d56330c6 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:57.316440 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d434020-1cf9-4948-a9ae-fde7d56330c6 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:57.316812 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d434020-1cf9-4948-a9ae-fde7d56330c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:57.326928 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d434020-1cf9-4948-a9ae-fde7d56330c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:57.328362 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d434020-1cf9-4948-a9ae-fde7d56330c6 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:57.345466 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-8d434020-1cf9-4948-a9ae-fde7d56330c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:44:57.345466 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:57.345466 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:57.345466 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:57.345466 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:57.345466 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:57.345466 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:57.345466 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:57.345466 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:57.345466 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:57.345466 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:57.345466 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:57.345466 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:44:57.345466 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:44:57.345466 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:44:57.346542 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:44:57.346542 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:57.346542 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:57.346542 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:57.346542 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:57.346542 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:57.346542 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:57.346542 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:57.346542 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:44:57.346542 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:57.346542 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:57.346542 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:57.346542 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:57.346542 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:57.346542 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:57.346542 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:57.346542 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:57.353161 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:44:57.353161 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:44:57.353161 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:44:57.353161 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:44:57.353161 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 250/507] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:44:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:57.353161 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d434020-1cf9-4948-a9ae-fde7d56330c6 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:57.369951 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60517598-f8f1-47fc-9a2b-b5996771769b None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:57.401993 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e332f9-c7cd-40e6-bb25-d66f6076f44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:57.402654 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e332f9-c7cd-40e6-bb25-d66f6076f44a None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:57.402937 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e332f9-c7cd-40e6-bb25-d66f6076f44a None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:57.403214 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e332f9-c7cd-40e6-bb25-d66f6076f44a None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:57.404031 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e332f9-c7cd-40e6-bb25-d66f6076f44a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:57.404338 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e332f9-c7cd-40e6-bb25-d66f6076f44a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:57.406049 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e332f9-c7cd-40e6-bb25-d66f6076f44a None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:57.422436 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 258/508] 10.222.0.23 () {56 vars in 1137 bytes} [Fri May 6 14:44:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:57.423231 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e332f9-c7cd-40e6-bb25-d66f6076f44a None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:57.449226 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d434020-1cf9-4948-a9ae-fde7d56330c6 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:57.485656 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d339f4-10b2-4271-a4b7-b36553247541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:57.486438 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d339f4-10b2-4271-a4b7-b36553247541 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:57.486718 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d339f4-10b2-4271-a4b7-b36553247541 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:57.486991 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d339f4-10b2-4271-a4b7-b36553247541 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:57.497202 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d339f4-10b2-4271-a4b7-b36553247541 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:57.498625 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d339f4-10b2-4271-a4b7-b36553247541 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:57.515909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-a9d339f4-10b2-4271-a4b7-b36553247541 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:44:57.515909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:57.515909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:57.515909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:57.515909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:57.515909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:57.515909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:57.515909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:57.515909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:57.515909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:57.515909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:57.515909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:57.515909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:44:57.515909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:44:57.515909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:44:57.516971 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:44:57.516971 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:57.516971 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:57.516971 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:57.516971 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:57.516971 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:57.516971 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:57.516971 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:57.516971 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:44:57.516971 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:57.516971 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:57.516971 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:57.516971 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:57.516971 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:57.516971 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:57.516971 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:57.516971 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:57.517841 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:44:57.517841 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:44:57.517841 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:44:57.517841 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:44:57.517841 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 251/509] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:44:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:57.518432 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d339f4-10b2-4271-a4b7-b36553247541 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:57.518522 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e332f9-c7cd-40e6-bb25-d66f6076f44a None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:57.551250 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc026ea-1489-4bae-a1a1-1bcb9bf4462e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:57.551912 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc026ea-1489-4bae-a1a1-1bcb9bf4462e None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:57.552195 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc026ea-1489-4bae-a1a1-1bcb9bf4462e None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:57.552473 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc026ea-1489-4bae-a1a1-1bcb9bf4462e None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:57.553336 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc026ea-1489-4bae-a1a1-1bcb9bf4462e None admin] RBAC: Adding query filter params (name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:57.553639 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc026ea-1489-4bae-a1a1-1bcb9bf4462e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:57.555036 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc026ea-1489-4bae-a1a1-1bcb9bf4462e None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:57.562598 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 259/510] 10.222.0.23 () {56 vars in 1103 bytes} [Fri May 6 14:44:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:57.563381 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc026ea-1489-4bae-a1a1-1bcb9bf4462e None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:57.614099 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d339f4-10b2-4271-a4b7-b36553247541 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:57.646305 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed094f-fd0d-43bf-ae6b-bcccbff07fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:57.647689 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed094f-fd0d-43bf-ae6b-bcccbff07fb7 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:57.647980 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed094f-fd0d-43bf-ae6b-bcccbff07fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:57.648234 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed094f-fd0d-43bf-ae6b-bcccbff07fb7 None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:57.648985 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ed094f-fd0d-43bf-ae6b-bcccbff07fb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:57.649293 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ed094f-fd0d-43bf-ae6b-bcccbff07fb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:57.650670 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ed094f-fd0d-43bf-ae6b-bcccbff07fb7 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:57.657213 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc026ea-1489-4bae-a1a1-1bcb9bf4462e None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:57.662092 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 252/511] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:44:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:57.663053 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed094f-fd0d-43bf-ae6b-bcccbff07fb7 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:57.689316 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0bb57e-47a0-4d27-baa6-a846e9f95969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:57.689969 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0bb57e-47a0-4d27-baa6-a846e9f95969 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:57.690255 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0bb57e-47a0-4d27-baa6-a846e9f95969 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:57.690531 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0bb57e-47a0-4d27-baa6-a846e9f95969 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:57.691240 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0bb57e-47a0-4d27-baa6-a846e9f95969 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:57.691555 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0bb57e-47a0-4d27-baa6-a846e9f95969 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:57.692941 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0bb57e-47a0-4d27-baa6-a846e9f95969 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:57.704768 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 260/512] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:44:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:57.705498 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0bb57e-47a0-4d27-baa6-a846e9f95969 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:57.760749 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed094f-fd0d-43bf-ae6b-bcccbff07fb7 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:57.792619 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a74572-7257-43eb-834c-4724c191275c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:57.793261 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a74572-7257-43eb-834c-4724c191275c None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:57.793543 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a74572-7257-43eb-834c-4724c191275c None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:57.793813 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a74572-7257-43eb-834c-4724c191275c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:57.798677 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0bb57e-47a0-4d27-baa6-a846e9f95969 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:57.831348 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c65e55-2008-40a0-8bfd-628318b4bcfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:57.832002 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c65e55-2008-40a0-8bfd-628318b4bcfe None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:57.832288 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c65e55-2008-40a0-8bfd-628318b4bcfe None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:57.832571 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c65e55-2008-40a0-8bfd-628318b4bcfe None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:57.833323 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c65e55-2008-40a0-8bfd-628318b4bcfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:57.833627 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c65e55-2008-40a0-8bfd-628318b4bcfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:57.835015 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c65e55-2008-40a0-8bfd-628318b4bcfe None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:57.845810 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 261/513] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:44:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:57.846603 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c65e55-2008-40a0-8bfd-628318b4bcfe None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:57.857681 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a74572-7257-43eb-834c-4724c191275c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:57.859119 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a74572-7257-43eb-834c-4724c191275c None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:57.941677 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c65e55-2008-40a0-8bfd-628318b4bcfe None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:57.952180 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 253/514] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:44:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 06 14:44:57.953036 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a74572-7257-43eb-834c-4724c191275c None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:57.973754 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a2cf9-3425-476b-afd6-e97a03ac9dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:57.974464 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a2cf9-3425-476b-afd6-e97a03ac9dbb None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:57.974747 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a2cf9-3425-476b-afd6-e97a03ac9dbb None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:57.975262 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a2cf9-3425-476b-afd6-e97a03ac9dbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:57.985394 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7a2cf9-3425-476b-afd6-e97a03ac9dbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:57.986842 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7a2cf9-3425-476b-afd6-e97a03ac9dbb None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:58.003829 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-2d7a2cf9-3425-476b-afd6-e97a03ac9dbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:44:58.003829 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:58.003829 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:58.003829 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:58.003829 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:58.003829 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:58.003829 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:58.003829 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:58.003829 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:58.003829 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:58.003829 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:58.003829 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:58.003829 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:44:58.003829 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:44:58.003829 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:44:58.004873 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:44:58.004873 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:58.004873 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:58.004873 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:58.004873 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:58.004873 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:58.004873 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:58.004873 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:58.004873 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:44:58.004873 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:58.004873 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:58.004873 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:58.004873 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:58.004873 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:58.004873 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:58.004873 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:58.004873 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:58.005905 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:44:58.005905 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:44:58.005905 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:44:58.005905 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:44:58.005905 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 262/515] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:44:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:58.006845 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a2cf9-3425-476b-afd6-e97a03ac9dbb None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:58.049162 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a74572-7257-43eb-834c-4724c191275c None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:58.082080 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10851d6e-8d7b-470e-a4ad-e11a2bc352b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:58.082861 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10851d6e-8d7b-470e-a4ad-e11a2bc352b3 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:58.083141 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10851d6e-8d7b-470e-a4ad-e11a2bc352b3 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:58.083413 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10851d6e-8d7b-470e-a4ad-e11a2bc352b3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:58.090143 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10851d6e-8d7b-470e-a4ad-e11a2bc352b3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:58.091556 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10851d6e-8d7b-470e-a4ad-e11a2bc352b3 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:58.099799 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a2cf9-3425-476b-afd6-e97a03ac9dbb None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:58.105689 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-10851d6e-8d7b-470e-a4ad-e11a2bc352b3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 06 14:44:58.105689 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:58.105689 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:58.105689 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:58.105689 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:58.105689 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:58.105689 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:58.105689 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:58.105689 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:58.105689 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:58.105689 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:58.105689 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:58.105689 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 06 14:44:58.105689 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_user(user_id) May 06 14:44:58.105689 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 06 14:44:58.105689 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 06 14:44:58.106571 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:58.106571 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:58.106571 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 06 14:44:58.106571 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:44:58.106571 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 06 14:44:58.106571 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:44:58.106571 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:58.106571 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:58.106571 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:58.106571 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:58.106571 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:58.106571 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:44:58.106571 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:58.106571 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:58.106571 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:58.106571 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:58.106571 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:58.106571 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:58.107487 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:58.107487 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:58.107487 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 06 14:44:58.107487 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 06 14:44:58.107487 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 06 14:44:58.107487 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 06 14:44:58.107487 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 06 14:44:58.107487 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 06 14:44:58.107487 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 06 14:44:58.107487 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:44:58.107487 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 254/516] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:44:57 2022] GET /identity/v3/users/glance => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:58.108063 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10851d6e-8d7b-470e-a4ad-e11a2bc352b3 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:58.132892 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a7c02a-afa0-4256-80a3-6db1d67bc68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:58.133686 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7c02a-afa0-4256-80a3-6db1d67bc68b None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:58.133973 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7c02a-afa0-4256-80a3-6db1d67bc68b None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:58.134262 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7c02a-afa0-4256-80a3-6db1d67bc68b None admin] PATH_INFO: `/v3/projects/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:58.140737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a7c02a-afa0-4256-80a3-6db1d67bc68b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:58.142163 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a7c02a-afa0-4256-80a3-6db1d67bc68b None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:58.156953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-99a7c02a-afa0-4256-80a3-6db1d67bc68b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:44:58.156953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:58.156953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:58.156953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:58.156953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:58.156953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:58.156953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:58.156953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:58.156953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:58.156953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:58.156953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:58.156953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:58.156953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 06 14:44:58.156953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(project_id) May 06 14:44:58.156953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 06 14:44:58.157835 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 06 14:44:58.157835 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:58.157835 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:58.157835 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:58.157835 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:58.157835 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:58.157835 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:58.157835 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:58.157835 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:44:58.157835 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:58.157835 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:58.157835 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:58.157835 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:58.157835 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:58.157835 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:58.157835 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:58.157835 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:58.164043 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 06 14:44:58.164043 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 06 14:44:58.164043 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 06 14:44:58.164043 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 06 14:44:58.164043 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 06 14:44:58.164043 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 06 14:44:58.164043 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:44:58.164043 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:44:58.164043 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 263/517] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:44:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:58.164043 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a7c02a-afa0-4256-80a3-6db1d67bc68b None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:58.203835 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10851d6e-8d7b-470e-a4ad-e11a2bc352b3 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:58.236000 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fce2ff9-25ee-486b-98bb-38a76b3aeeb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:58.236685 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce2ff9-25ee-486b-98bb-38a76b3aeeb6 None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:58.236974 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce2ff9-25ee-486b-98bb-38a76b3aeeb6 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:58.237253 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce2ff9-25ee-486b-98bb-38a76b3aeeb6 None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:58.237974 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fce2ff9-25ee-486b-98bb-38a76b3aeeb6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:58.239401 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fce2ff9-25ee-486b-98bb-38a76b3aeeb6 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:58.253251 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a7c02a-afa0-4256-80a3-6db1d67bc68b None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:58.258853 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 255/518] 10.222.0.23 () {60 vars in 1132 bytes} [Fri May 6 14:44:58 2022] POST /identity/v3/users => generated 247 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:44:58.259662 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fce2ff9-25ee-486b-98bb-38a76b3aeeb6 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:58.290152 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5592c03-4c74-4c9f-b4fe-02601bc9a771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:58.290977 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5592c03-4c74-4c9f-b4fe-02601bc9a771 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:58.291317 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5592c03-4c74-4c9f-b4fe-02601bc9a771 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:58.291607 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5592c03-4c74-4c9f-b4fe-02601bc9a771 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:58.302632 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5592c03-4c74-4c9f-b4fe-02601bc9a771 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:58.305284 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5592c03-4c74-4c9f-b4fe-02601bc9a771 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:58.324108 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-d5592c03-4c74-4c9f-b4fe-02601bc9a771 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:44:58.324108 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:58.324108 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:58.324108 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:58.324108 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:58.324108 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:58.324108 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:58.324108 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:58.324108 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:58.324108 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:58.324108 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:58.324108 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:58.324108 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:44:58.324108 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:44:58.324108 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:44:58.325094 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:44:58.325094 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:58.325094 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:58.325094 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:58.325094 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:58.325094 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:58.325094 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:58.325094 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:58.325094 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:44:58.325094 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:58.325094 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:58.325094 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:58.325094 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:58.325094 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:58.325094 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:58.325094 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:58.325094 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:58.325924 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:44:58.325924 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:44:58.325924 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:44:58.325924 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:44:58.325924 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 264/519] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:44:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:58.326721 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5592c03-4c74-4c9f-b4fe-02601bc9a771 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:58.374271 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fce2ff9-25ee-486b-98bb-38a76b3aeeb6 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:58.410620 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8334bcd-aba9-44c1-b9fc-91ae20bbdc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:58.411296 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8334bcd-aba9-44c1-b9fc-91ae20bbdc59 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:58.411581 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8334bcd-aba9-44c1-b9fc-91ae20bbdc59 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:58.411857 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8334bcd-aba9-44c1-b9fc-91ae20bbdc59 None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:58.412755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8334bcd-aba9-44c1-b9fc-91ae20bbdc59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:58.413069 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8334bcd-aba9-44c1-b9fc-91ae20bbdc59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:58.414484 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8334bcd-aba9-44c1-b9fc-91ae20bbdc59 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:58.427972 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 256/520] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:44:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:58.429081 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8334bcd-aba9-44c1-b9fc-91ae20bbdc59 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:58.445861 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5592c03-4c74-4c9f-b4fe-02601bc9a771 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:58.485744 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227d2d0a-958c-4afd-b225-127dee8b4204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:58.486624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d2d0a-958c-4afd-b225-127dee8b4204 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:58.486956 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d2d0a-958c-4afd-b225-127dee8b4204 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:58.487331 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d2d0a-958c-4afd-b225-127dee8b4204 None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:58.488284 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227d2d0a-958c-4afd-b225-127dee8b4204 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:58.488743 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227d2d0a-958c-4afd-b225-127dee8b4204 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:58.490278 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227d2d0a-958c-4afd-b225-127dee8b4204 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:58.510289 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 265/521] 10.222.0.23 () {56 vars in 1137 bytes} [Fri May 6 14:44:58 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:58.511209 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227d2d0a-958c-4afd-b225-127dee8b4204 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:58.544629 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8334bcd-aba9-44c1-b9fc-91ae20bbdc59 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:58.589695 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a2617-e8fe-4fe3-a24e-c7e59abdb229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:58.590369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a2617-e8fe-4fe3-a24e-c7e59abdb229 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:58.590652 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a2617-e8fe-4fe3-a24e-c7e59abdb229 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:58.590929 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a2617-e8fe-4fe3-a24e-c7e59abdb229 None admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:58.591722 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a2617-e8fe-4fe3-a24e-c7e59abdb229 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:58.592032 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a2617-e8fe-4fe3-a24e-c7e59abdb229 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:58.593505 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a2617-e8fe-4fe3-a24e-c7e59abdb229 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:58.616324 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 257/522] 10.222.0.23 () {56 vars in 1148 bytes} [Fri May 6 14:44:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:58.616324 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a2617-e8fe-4fe3-a24e-c7e59abdb229 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:58.622295 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227d2d0a-958c-4afd-b225-127dee8b4204 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:58.655498 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ec498-67f1-4fab-9dac-85ccf71f7311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:58.656428 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ec498-67f1-4fab-9dac-85ccf71f7311 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:58.656740 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ec498-67f1-4fab-9dac-85ccf71f7311 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:58.657018 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ec498-67f1-4fab-9dac-85ccf71f7311 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:58.657736 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ec498-67f1-4fab-9dac-85ccf71f7311 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:58.658043 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ec498-67f1-4fab-9dac-85ccf71f7311 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:58.659397 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ec498-67f1-4fab-9dac-85ccf71f7311 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:58.674601 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 266/523] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:44:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:58.674601 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ec498-67f1-4fab-9dac-85ccf71f7311 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:58.717441 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a2617-e8fe-4fe3-a24e-c7e59abdb229 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:58.750973 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a894befb-de09-42a9-84f0-9e61dc467f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:58.751718 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894befb-de09-42a9-84f0-9e61dc467f13 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:58.752001 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894befb-de09-42a9-84f0-9e61dc467f13 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:58.752278 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894befb-de09-42a9-84f0-9e61dc467f13 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:58.760034 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a894befb-de09-42a9-84f0-9e61dc467f13 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:58.761498 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a894befb-de09-42a9-84f0-9e61dc467f13 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:58.767456 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ec498-67f1-4fab-9dac-85ccf71f7311 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:58.776184 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-a894befb-de09-42a9-84f0-9e61dc467f13 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 06 14:44:58.776184 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:58.776184 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:58.776184 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:58.776184 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:58.776184 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:58.776184 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:58.776184 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:58.776184 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:58.776184 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:58.776184 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:58.776184 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:58.776184 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 06 14:44:58.776184 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_user(user_id) May 06 14:44:58.776184 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 06 14:44:58.776184 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 06 14:44:58.777389 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:58.777389 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:58.777389 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 06 14:44:58.777389 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:44:58.777389 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 06 14:44:58.777389 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:44:58.777389 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:58.777389 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:58.777389 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:58.777389 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:58.777389 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:58.777389 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:44:58.777389 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:58.777389 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:58.777389 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:58.777389 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:58.777389 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:58.777389 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:58.778302 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:58.778302 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:58.778302 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 06 14:44:58.778302 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 06 14:44:58.778302 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 06 14:44:58.778302 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 06 14:44:58.778302 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 06 14:44:58.778302 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 06 14:44:58.778302 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 06 14:44:58.778302 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:44:58.778302 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 258/524] 10.222.0.23 () {56 vars in 1102 bytes} [Fri May 6 14:44:58 2022] GET /identity/v3/users/neutron => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:58.778302 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a894befb-de09-42a9-84f0-9e61dc467f13 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:58.804430 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde18756-1faa-459f-abef-daa89a384482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:58.805116 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde18756-1faa-459f-abef-daa89a384482 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:58.805403 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde18756-1faa-459f-abef-daa89a384482 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:58.805681 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde18756-1faa-459f-abef-daa89a384482 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:58.871216 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde18756-1faa-459f-abef-daa89a384482 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:58.872715 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde18756-1faa-459f-abef-daa89a384482 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:58.884837 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a894befb-de09-42a9-84f0-9e61dc467f13 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:58.917680 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a1cfd-4cf4-4e96-a89d-c0032e6c3f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:58.918507 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a1cfd-4cf4-4e96-a89d-c0032e6c3f44 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:58.918794 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a1cfd-4cf4-4e96-a89d-c0032e6c3f44 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:58.919071 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a1cfd-4cf4-4e96-a89d-c0032e6c3f44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:58.919896 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209a1cfd-4cf4-4e96-a89d-c0032e6c3f44 None admin] RBAC: Adding query filter params (role.id=f5641817ee56438496afc6135d37a4c0, scope.project.id=d5398feee6eb490cb49bb261448a1a9b, user.id=fc8cf09642ec4e10a80badcf8b585ef8) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:58.920211 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209a1cfd-4cf4-4e96-a89d-c0032e6c3f44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:58.921681 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209a1cfd-4cf4-4e96-a89d-c0032e6c3f44 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:58.936957 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 259/525] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:44:58 2022] GET /identity/v3/role_assignments?user.id=fc8cf09642ec4e10a80badcf8b585ef8&scope.project.id=d5398feee6eb490cb49bb261448a1a9b&role.id=f5641817ee56438496afc6135d37a4c0 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:58.936957 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a1cfd-4cf4-4e96-a89d-c0032e6c3f44 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:58.974650 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 267/526] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:44:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 06 14:44:58.976620 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde18756-1faa-459f-abef-daa89a384482 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:59.046292 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a1cfd-4cf4-4e96-a89d-c0032e6c3f44 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:59.083119 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde18756-1faa-459f-abef-daa89a384482 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:59.085339 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19bb98-6952-4fff-8992-d245d5c7ca9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:59.086220 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19bb98-6952-4fff-8992-d245d5c7ca9b None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:59.086502 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19bb98-6952-4fff-8992-d245d5c7ca9b None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:59.086777 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19bb98-6952-4fff-8992-d245d5c7ca9b None admin] PATH_INFO: `/v3/projects/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:59.094555 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb19bb98-6952-4fff-8992-d245d5c7ca9b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:59.096120 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb19bb98-6952-4fff-8992-d245d5c7ca9b None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:59.112241 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-bb19bb98-6952-4fff-8992-d245d5c7ca9b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:44:59.112241 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:59.112241 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:59.112241 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:59.112241 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:59.112241 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:59.112241 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:59.112241 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:59.112241 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:59.112241 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:59.112241 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:59.112241 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:59.112241 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 06 14:44:59.112241 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(project_id) May 06 14:44:59.112241 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 06 14:44:59.113192 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 06 14:44:59.113192 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:59.113192 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:59.113192 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:59.113192 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:59.113192 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:59.113192 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:59.113192 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:59.113192 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:44:59.113192 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:59.113192 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:59.113192 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:59.113192 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:59.113192 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:59.113192 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:59.113192 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:59.113192 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:59.114013 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 06 14:44:59.114013 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 06 14:44:59.114013 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 06 14:44:59.114013 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 06 14:44:59.114013 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 06 14:44:59.114013 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 06 14:44:59.114013 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:44:59.114013 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:44:59.114013 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 260/527] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:44:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:59.114937 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19bb98-6952-4fff-8992-d245d5c7ca9b None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:59.125236 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a7617-db69-4cc7-85f9-dccdc276e433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:59.125917 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a7617-db69-4cc7-85f9-dccdc276e433 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:59.126199 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a7617-db69-4cc7-85f9-dccdc276e433 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:59.126472 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a7617-db69-4cc7-85f9-dccdc276e433 None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:59.127312 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8a7617-db69-4cc7-85f9-dccdc276e433 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:59.127626 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8a7617-db69-4cc7-85f9-dccdc276e433 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:59.129113 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8a7617-db69-4cc7-85f9-dccdc276e433 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:59.148149 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 268/528] 10.222.0.23 () {56 vars in 1139 bytes} [Fri May 6 14:44:58 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:59.149038 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a7617-db69-4cc7-85f9-dccdc276e433 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:59.227194 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19bb98-6952-4fff-8992-d245d5c7ca9b None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:59.252445 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a7617-db69-4cc7-85f9-dccdc276e433 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:59.262371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f85fe1-0926-4715-8875-fd50b34ebf94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:59.263165 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f85fe1-0926-4715-8875-fd50b34ebf94 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:59.263525 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f85fe1-0926-4715-8875-fd50b34ebf94 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:59.263806 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f85fe1-0926-4715-8875-fd50b34ebf94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:59.275269 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f85fe1-0926-4715-8875-fd50b34ebf94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:59.276742 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f85fe1-0926-4715-8875-fd50b34ebf94 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:59.288353 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193b09e3-136e-457b-ba07-f7e71dc54034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:59.289042 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b09e3-136e-457b-ba07-f7e71dc54034 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:59.289324 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b09e3-136e-457b-ba07-f7e71dc54034 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:59.289598 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b09e3-136e-457b-ba07-f7e71dc54034 None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:59.290376 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193b09e3-136e-457b-ba07-f7e71dc54034 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:59.290684 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193b09e3-136e-457b-ba07-f7e71dc54034 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:59.292131 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193b09e3-136e-457b-ba07-f7e71dc54034 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:59.294996 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-41f85fe1-0926-4715-8875-fd50b34ebf94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:44:59.294996 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:59.294996 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:59.294996 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:59.294996 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:59.294996 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:59.294996 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:59.294996 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:59.294996 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:59.294996 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:59.294996 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:59.294996 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:59.294996 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:44:59.294996 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:44:59.294996 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:44:59.295810 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:44:59.295810 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:59.295810 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:59.295810 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:59.295810 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:59.295810 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:59.295810 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:59.295810 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:59.295810 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:44:59.295810 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:59.295810 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:59.295810 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:59.295810 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:59.295810 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:59.295810 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:59.295810 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:59.295810 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:59.296712 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:44:59.296712 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:44:59.296712 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:44:59.296712 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:44:59.300991 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 261/529] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:44:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:59.301845 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f85fe1-0926-4715-8875-fd50b34ebf94 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:59.315528 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 269/530] 10.222.0.23 () {56 vars in 1148 bytes} [Fri May 6 14:44:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:59.316300 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193b09e3-136e-457b-ba07-f7e71dc54034 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:59.403967 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f85fe1-0926-4715-8875-fd50b34ebf94 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:59.416198 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193b09e3-136e-457b-ba07-f7e71dc54034 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:59.439392 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d6feee-76e6-4d44-9827-f4c592c551c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:59.440163 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6feee-76e6-4d44-9827-f4c592c551c3 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:59.440452 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6feee-76e6-4d44-9827-f4c592c551c3 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:59.440767 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6feee-76e6-4d44-9827-f4c592c551c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:59.451473 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c61ee-bc05-4bdd-b86b-64feafbbf4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:59.452137 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c61ee-bc05-4bdd-b86b-64feafbbf4f5 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:59.452420 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c61ee-bc05-4bdd-b86b-64feafbbf4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:59.452727 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c61ee-bc05-4bdd-b86b-64feafbbf4f5 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:59.453668 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18c61ee-bc05-4bdd-b86b-64feafbbf4f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:59.453979 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18c61ee-bc05-4bdd-b86b-64feafbbf4f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:59.455341 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18c61ee-bc05-4bdd-b86b-64feafbbf4f5 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:59.469186 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 270/531] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:44:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:59.470018 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c61ee-bc05-4bdd-b86b-64feafbbf4f5 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:59.516851 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d6feee-76e6-4d44-9827-f4c592c551c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:59.518380 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d6feee-76e6-4d44-9827-f4c592c551c3 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:59.573272 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c61ee-bc05-4bdd-b86b-64feafbbf4f5 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:59.606998 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e715dd1-ed03-4d4d-a6e1-6a8fb46b0ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:59.607658 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e715dd1-ed03-4d4d-a6e1-6a8fb46b0ed7 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:59.607942 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e715dd1-ed03-4d4d-a6e1-6a8fb46b0ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:59.608222 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e715dd1-ed03-4d4d-a6e1-6a8fb46b0ed7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:59.609082 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e715dd1-ed03-4d4d-a6e1-6a8fb46b0ed7 None admin] RBAC: Adding query filter params (role.id=f5641817ee56438496afc6135d37a4c0, scope.project.id=d5398feee6eb490cb49bb261448a1a9b, user.id=6409b68b4832421ba590d7bc53131f7c) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:44:59.609395 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e715dd1-ed03-4d4d-a6e1-6a8fb46b0ed7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:59.610823 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e715dd1-ed03-4d4d-a6e1-6a8fb46b0ed7 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:59.620870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 271/532] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:44:59 2022] GET /identity/v3/role_assignments?user.id=6409b68b4832421ba590d7bc53131f7c&scope.project.id=d5398feee6eb490cb49bb261448a1a9b&role.id=f5641817ee56438496afc6135d37a4c0 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:44:59.621790 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e715dd1-ed03-4d4d-a6e1-6a8fb46b0ed7 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:59.628874 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 262/533] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:44:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 06 14:44:59.629887 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d6feee-76e6-4d44-9827-f4c592c551c3 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:59.737923 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d6feee-76e6-4d44-9827-f4c592c551c3 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:59.778310 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b29105-97e7-4c44-b305-49e62b4c8fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:59.779159 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b29105-97e7-4c44-b305-49e62b4c8fd3 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:59.779446 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b29105-97e7-4c44-b305-49e62b4c8fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:59.779722 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b29105-97e7-4c44-b305-49e62b4c8fd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:59.791979 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b29105-97e7-4c44-b305-49e62b4c8fd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:59.793551 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b29105-97e7-4c44-b305-49e62b4c8fd3 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:59.813110 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-a4b29105-97e7-4c44-b305-49e62b4c8fd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:44:59.813110 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:59.813110 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:59.813110 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:59.813110 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:59.813110 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:59.813110 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:59.813110 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:59.813110 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:59.813110 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:59.813110 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:59.813110 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:59.813110 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:44:59.813110 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:44:59.813110 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:44:59.815141 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:44:59.815141 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:59.815141 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:59.815141 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:59.815141 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:59.815141 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:59.815141 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:59.815141 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:59.815141 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:44:59.815141 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:59.815141 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:59.815141 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:59.815141 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:59.815141 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:59.815141 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:59.815141 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:59.815141 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:59.817282 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:44:59.817282 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:44:59.817282 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:44:59.817282 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:44:59.817282 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 263/534] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:44:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:59.821859 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b29105-97e7-4c44-b305-49e62b4c8fd3 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:59.835689 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e715dd1-ed03-4d4d-a6e1-6a8fb46b0ed7 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:44:59.883011 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea25f1-b0ba-4c81-ad6d-176d2a4241be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:44:59.884163 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea25f1-b0ba-4c81-ad6d-176d2a4241be None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:44:59.884671 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea25f1-b0ba-4c81-ad6d-176d2a4241be None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:44:59.885163 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea25f1-b0ba-4c81-ad6d-176d2a4241be None admin] PATH_INFO: `/v3/projects/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:44:59.894466 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea25f1-b0ba-4c81-ad6d-176d2a4241be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:44:59.896204 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea25f1-b0ba-4c81-ad6d-176d2a4241be None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:44:59.913541 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-98ea25f1-b0ba-4c81-ad6d-176d2a4241be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:44:59.913541 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:44:59.913541 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:44:59.913541 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:44:59.913541 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:44:59.913541 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:44:59.913541 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:44:59.913541 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:44:59.913541 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:44:59.913541 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:44:59.913541 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:44:59.913541 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:44:59.913541 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 06 14:44:59.913541 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(project_id) May 06 14:44:59.913541 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 06 14:44:59.914464 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 06 14:44:59.914464 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:44:59.914464 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:44:59.914464 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:44:59.914464 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:44:59.914464 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:44:59.914464 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:44:59.914464 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:44:59.914464 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:44:59.914464 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:44:59.914464 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:44:59.914464 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:44:59.914464 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:44:59.914464 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:44:59.914464 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:44:59.914464 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:44:59.914464 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:44:59.915275 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 06 14:44:59.915275 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 06 14:44:59.915275 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 06 14:44:59.915275 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 06 14:44:59.915275 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 06 14:44:59.915275 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 06 14:44:59.915275 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:44:59.915275 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:44:59.917753 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 272/535] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:44:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:44:59.927286 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea25f1-b0ba-4c81-ad6d-176d2a4241be None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:44:59.959014 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b29105-97e7-4c44-b305-49e62b4c8fd3 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:00.027355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d965e7c-d755-44e5-a990-f7c280c3ab6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:00.028369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d965e7c-d755-44e5-a990-f7c280c3ab6d None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:00.029874 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d965e7c-d755-44e5-a990-f7c280c3ab6d None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:00.030349 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d965e7c-d755-44e5-a990-f7c280c3ab6d None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:00.031608 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d965e7c-d755-44e5-a990-f7c280c3ab6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:00.032270 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d965e7c-d755-44e5-a990-f7c280c3ab6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:00.035000 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d965e7c-d755-44e5-a990-f7c280c3ab6d None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:00.060895 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 264/536] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:44:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:00.063560 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d965e7c-d755-44e5-a990-f7c280c3ab6d None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:00.116961 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea25f1-b0ba-4c81-ad6d-176d2a4241be None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:00.181159 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d965e7c-d755-44e5-a990-f7c280c3ab6d None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:00.194399 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e78c39-9134-4ec2-87f2-f33596f8874f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:00.195144 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e78c39-9134-4ec2-87f2-f33596f8874f None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:00.195362 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e78c39-9134-4ec2-87f2-f33596f8874f None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:00.195644 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e78c39-9134-4ec2-87f2-f33596f8874f None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:00.196435 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e78c39-9134-4ec2-87f2-f33596f8874f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:00.196781 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e78c39-9134-4ec2-87f2-f33596f8874f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:00.198222 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e78c39-9134-4ec2-87f2-f33596f8874f None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:00.219555 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c28c363-ef90-4f63-bfa0-7ba1a22cc58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:00.220760 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c28c363-ef90-4f63-bfa0-7ba1a22cc58d None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:00.221260 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c28c363-ef90-4f63-bfa0-7ba1a22cc58d None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:00.221728 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c28c363-ef90-4f63-bfa0-7ba1a22cc58d None admin] PATH_INFO: `/v3/projects/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:00.222893 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 273/537] 10.222.0.23 () {56 vars in 1148 bytes} [Fri May 6 14:44:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:00.229815 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e78c39-9134-4ec2-87f2-f33596f8874f None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:00.231374 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c28c363-ef90-4f63-bfa0-7ba1a22cc58d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:00.233200 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c28c363-ef90-4f63-bfa0-7ba1a22cc58d None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:00.248358 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-1c28c363-ef90-4f63-bfa0-7ba1a22cc58d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:00.248358 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:00.248358 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:00.248358 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:00.248358 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:00.248358 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:00.248358 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:00.248358 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:00.248358 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:00.248358 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:00.248358 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:00.248358 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:00.248358 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 06 14:45:00.248358 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(project_id) May 06 14:45:00.248358 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 06 14:45:00.249285 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 06 14:45:00.249285 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:00.249285 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:00.249285 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:00.249285 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:00.249285 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:00.249285 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:00.249285 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:00.249285 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:00.249285 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:00.249285 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:00.249285 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:00.249285 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:00.249285 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:00.249285 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:00.249285 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:00.249285 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:00.250123 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 06 14:45:00.250123 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 06 14:45:00.250123 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 06 14:45:00.250123 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 06 14:45:00.250123 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 06 14:45:00.250123 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 06 14:45:00.250123 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:00.250123 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:00.252574 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 265/538] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:45:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:00.261141 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c28c363-ef90-4f63-bfa0-7ba1a22cc58d None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:00.368468 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e78c39-9134-4ec2-87f2-f33596f8874f None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:00.408820 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659878d1-032c-4755-a7a3-5113739719ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:00.409481 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659878d1-032c-4755-a7a3-5113739719ca None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:00.409762 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659878d1-032c-4755-a7a3-5113739719ca None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:00.410035 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659878d1-032c-4755-a7a3-5113739719ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:00.410860 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659878d1-032c-4755-a7a3-5113739719ca None admin] RBAC: Adding query filter params (role.id=f5641817ee56438496afc6135d37a4c0, scope.project.id=d5398feee6eb490cb49bb261448a1a9b, user.id=d9be93f9ff7e46babc8ee87b50b9c5eb) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:00.411169 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659878d1-032c-4755-a7a3-5113739719ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:00.412923 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659878d1-032c-4755-a7a3-5113739719ca None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:00.418910 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 274/539] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:45:00 2022] GET /identity/v3/role_assignments?user.id=d9be93f9ff7e46babc8ee87b50b9c5eb&scope.project.id=d5398feee6eb490cb49bb261448a1a9b&role.id=f5641817ee56438496afc6135d37a4c0 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:00.429569 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c28c363-ef90-4f63-bfa0-7ba1a22cc58d None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:00.472433 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917c741-d61a-4b35-adbb-04f4a7f2363c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:00.473292 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917c741-d61a-4b35-adbb-04f4a7f2363c None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:00.473577 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917c741-d61a-4b35-adbb-04f4a7f2363c None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:00.473855 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917c741-d61a-4b35-adbb-04f4a7f2363c None admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:00.474776 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b917c741-d61a-4b35-adbb-04f4a7f2363c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:00.475147 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b917c741-d61a-4b35-adbb-04f4a7f2363c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:00.476578 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b917c741-d61a-4b35-adbb-04f4a7f2363c None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:00.493366 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 266/540] 10.222.0.23 () {56 vars in 1148 bytes} [Fri May 6 14:45:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:00.498865 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659878d1-032c-4755-a7a3-5113739719ca None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:00.658874 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659878d1-032c-4755-a7a3-5113739719ca None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:00.703863 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090b8001-3b1f-4b99-8d68-7b778ae24d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:00.704891 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b8001-3b1f-4b99-8d68-7b778ae24d34 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:00.705373 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b8001-3b1f-4b99-8d68-7b778ae24d34 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:00.705838 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b8001-3b1f-4b99-8d68-7b778ae24d34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:00.706888 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090b8001-3b1f-4b99-8d68-7b778ae24d34 None admin] RBAC: Adding query filter params (role.id=f5641817ee56438496afc6135d37a4c0, scope.project.id=d5398feee6eb490cb49bb261448a1a9b, user.id=4dcd1542d8f643c4996142a88b9b6abb) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:00.707402 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090b8001-3b1f-4b99-8d68-7b778ae24d34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:00.709068 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090b8001-3b1f-4b99-8d68-7b778ae24d34 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:00.718973 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 275/541] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:45:00 2022] GET /identity/v3/role_assignments?user.id=4dcd1542d8f643c4996142a88b9b6abb&scope.project.id=d5398feee6eb490cb49bb261448a1a9b&role.id=f5641817ee56438496afc6135d37a4c0 => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:00.841331 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4612c-d10d-48a5-b578-d4b3e1abbed9 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:00.841677 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4612c-d10d-48a5-b578-d4b3e1abbed9 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:00.841913 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4612c-d10d-48a5-b578-d4b3e1abbed9 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:00.843646 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 267/542] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:45:00.849511 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0fa02-f3b4-49d4-ad6e-042f7544e866 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:00.850453 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0fa02-f3b4-49d4-ad6e-042f7544e866 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:00.850842 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0fa02-f3b4-49d4-ad6e-042f7544e866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:00.874615 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-3de0fa02-f3b4-49d4-ad6e-042f7544e866 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:00.875813 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-3de0fa02-f3b4-49d4-ad6e-042f7544e866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:00.958839 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 276/543] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:45:00.985423 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce4612c-d10d-48a5-b578-d4b3e1abbed9 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:01.093202 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce4612c-d10d-48a5-b578-d4b3e1abbed9 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:01.128469 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb6c80e-f643-47dc-8c58-4b16dd1658f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:01.129302 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb6c80e-f643-47dc-8c58-4b16dd1658f8 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:01.129582 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb6c80e-f643-47dc-8c58-4b16dd1658f8 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:01.129856 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb6c80e-f643-47dc-8c58-4b16dd1658f8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:01.141722 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb6c80e-f643-47dc-8c58-4b16dd1658f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:45:01.150547 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-2bb6c80e-f643-47dc-8c58-4b16dd1658f8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 06 14:45:01.150547 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:01.150547 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:01.150547 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:01.150547 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:01.150547 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:01.150547 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:01.150547 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:01.150547 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:01.150547 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:01.150547 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:01.150547 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:01.150547 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:45:01.150547 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:45:01.150547 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:45:01.150547 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:45:01.151405 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:45:01.151405 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:45:01.151405 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:45:01.151405 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:45:01.151405 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:01.151405 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:01.151405 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:01.151405 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:01.151405 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:01.151405 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:01.151405 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:01.151405 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:01.151405 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:01.151405 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:01.151405 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:01.151405 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:01.152190 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:01.152190 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:01.152190 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:01.152190 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:01.152190 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:45:01.152190 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:45:01.152190 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:45:01.152190 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:45:01.152190 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:45:01.152190 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:45:01.152190 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 06 14:45:01.152190 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:01.152190 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 268/544] 10.222.0.23 () {56 vars in 1102 bytes} [Fri May 6 14:45:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:01.160107 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de0fa02-f3b4-49d4-ad6e-042f7544e866 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:01.260429 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de0fa02-f3b4-49d4-ad6e-042f7544e866 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:01.295282 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d360b1a-3815-4ca6-ac17-847470eb9a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:01.295949 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d360b1a-3815-4ca6-ac17-847470eb9a61 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:01.296233 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d360b1a-3815-4ca6-ac17-847470eb9a61 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:01.296509 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d360b1a-3815-4ca6-ac17-847470eb9a61 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:01.297275 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d360b1a-3815-4ca6-ac17-847470eb9a61 None admin] RBAC: Adding query filter params (name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:01.297581 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d360b1a-3815-4ca6-ac17-847470eb9a61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:01.298930 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d360b1a-3815-4ca6-ac17-847470eb9a61 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:01.307331 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 277/545] 10.222.0.23 () {56 vars in 1103 bytes} [Fri May 6 14:45:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:01.312721 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb6c80e-f643-47dc-8c58-4b16dd1658f8 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:01.416458 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb6c80e-f643-47dc-8c58-4b16dd1658f8 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:01.455625 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b2585a-8a8d-43de-90e7-59ea686c2374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:01.456433 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2585a-8a8d-43de-90e7-59ea686c2374 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:01.456748 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2585a-8a8d-43de-90e7-59ea686c2374 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:01.457028 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2585a-8a8d-43de-90e7-59ea686c2374 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:01.470077 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b2585a-8a8d-43de-90e7-59ea686c2374 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:01.471547 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b2585a-8a8d-43de-90e7-59ea686c2374 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:01.489910 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-32b2585a-8a8d-43de-90e7-59ea686c2374 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:01.489910 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:01.489910 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:01.489910 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:01.489910 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:01.489910 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:01.489910 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:01.489910 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:01.489910 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:01.489910 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:01.489910 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:01.489910 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:01.489910 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:01.489910 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:01.489910 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:01.491824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:01.491824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:01.491824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:01.491824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:01.491824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:01.491824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:01.491824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:01.491824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:01.491824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:01.491824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:01.491824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:01.491824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:01.491824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:01.491824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:01.491824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:01.491824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:01.491824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:01.492688 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:01.492688 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:01.492688 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:01.492688 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:01.492688 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 269/546] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:01.497810 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d360b1a-3815-4ca6-ac17-847470eb9a61 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:01.636895 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d360b1a-3815-4ca6-ac17-847470eb9a61 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:01.659296 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddef50a-efb5-4e20-9901-5ab477bcc622 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:01.659589 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddef50a-efb5-4e20-9901-5ab477bcc622 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:01.659869 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddef50a-efb5-4e20-9901-5ab477bcc622 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:01.661910 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 270/547] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:45:01.672671 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a1fa4-06a9-463e-8268-5fdaef1a1dd4 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:01.673056 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a1fa4-06a9-463e-8268-5fdaef1a1dd4 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:01.673244 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a1fa4-06a9-463e-8268-5fdaef1a1dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:01.679214 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35772baf-34df-47b7-9014-7712efdca25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:01.679873 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35772baf-34df-47b7-9014-7712efdca25a None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:01.680160 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35772baf-34df-47b7-9014-7712efdca25a None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:01.680442 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35772baf-34df-47b7-9014-7712efdca25a None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:01.681199 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35772baf-34df-47b7-9014-7712efdca25a None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:01.681507 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35772baf-34df-47b7-9014-7712efdca25a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:01.682885 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35772baf-34df-47b7-9014-7712efdca25a None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:01.695609 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 278/548] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:01.703747 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-128a1fa4-06a9-463e-8268-5fdaef1a1dd4 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:01.705022 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-128a1fa4-06a9-463e-8268-5fdaef1a1dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:01.708857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35772baf-34df-47b7-9014-7712efdca25a None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:01.817483 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 271/549] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 06 14:45:01.854175 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a1fa4-06a9-463e-8268-5fdaef1a1dd4 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:01.859170 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35772baf-34df-47b7-9014-7712efdca25a None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:01.938871 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3430b2-ea71-439b-97ab-55439f8215e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:01.939602 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3430b2-ea71-439b-97ab-55439f8215e7 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:01.939884 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3430b2-ea71-439b-97ab-55439f8215e7 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:01.940158 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3430b2-ea71-439b-97ab-55439f8215e7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:01.957752 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3430b2-ea71-439b-97ab-55439f8215e7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:01.958794 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a1fa4-06a9-463e-8268-5fdaef1a1dd4 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:01.959909 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3430b2-ea71-439b-97ab-55439f8215e7 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:01.990073 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-2e3430b2-ea71-439b-97ab-55439f8215e7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 06 14:45:01.990073 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:01.990073 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:01.990073 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:01.990073 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:01.990073 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:01.990073 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:01.990073 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:01.990073 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:01.990073 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:01.990073 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:01.990073 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:01.990073 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 06 14:45:01.990073 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_user(user_id) May 06 14:45:01.990073 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 06 14:45:01.990073 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 06 14:45:01.991731 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:01.991731 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:01.991731 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 06 14:45:01.991731 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:01.991731 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 06 14:45:01.991731 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:01.991731 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:01.991731 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:01.991731 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:01.991731 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:01.991731 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:01.991731 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:01.991731 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:01.991731 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:01.991731 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:01.991731 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:01.991731 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:01.991731 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:01.992624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:01.992624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:01.992624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 06 14:45:01.992624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 06 14:45:01.992624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 06 14:45:01.992624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 06 14:45:01.992624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 06 14:45:01.992624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 06 14:45:01.992624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 06 14:45:01.992624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:01.992624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 279/550] 10.222.0.23 () {56 vars in 1096 bytes} [Fri May 6 14:45:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:01.998490 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3430b2-ea71-439b-97ab-55439f8215e7 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:01.999929 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2784eb-11b5-4d5c-8d6b-40703141e3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:02.000771 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2784eb-11b5-4d5c-8d6b-40703141e3dc None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:02.001168 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2784eb-11b5-4d5c-8d6b-40703141e3dc None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:02.001448 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2784eb-11b5-4d5c-8d6b-40703141e3dc None admin] PATH_INFO: `/v3/roles/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:02.013500 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2784eb-11b5-4d5c-8d6b-40703141e3dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:45:02.021368 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-0a2784eb-11b5-4d5c-8d6b-40703141e3dc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 06 14:45:02.021368 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:02.021368 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:02.021368 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:02.021368 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:02.021368 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:02.021368 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:02.021368 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:02.021368 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:02.021368 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:02.021368 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:02.021368 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:02.021368 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:45:02.021368 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:45:02.021368 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:45:02.021368 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:45:02.023287 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:45:02.023287 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:45:02.023287 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:45:02.023287 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:45:02.023287 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:02.023287 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:02.023287 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:02.023287 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:02.023287 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:02.023287 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:02.023287 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:02.023287 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:02.023287 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:02.023287 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:02.023287 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:02.023287 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:02.024076 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:02.024076 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:02.024076 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:02.024076 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:02.024076 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:45:02.024076 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:45:02.024076 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:45:02.024076 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:45:02.024076 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:45:02.024076 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:45:02.024076 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 06 14:45:02.024076 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:02.024076 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 272/551] 10.222.0.23 () {56 vars in 1102 bytes} [Fri May 6 14:45:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:45:02.029378 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2784eb-11b5-4d5c-8d6b-40703141e3dc None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:02.133095 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2784eb-11b5-4d5c-8d6b-40703141e3dc None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:02.144535 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3430b2-ea71-439b-97ab-55439f8215e7 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:02.167761 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03605f6f-48d3-4119-91ea-8664805593e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:02.168649 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03605f6f-48d3-4119-91ea-8664805593e3 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:02.169140 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03605f6f-48d3-4119-91ea-8664805593e3 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:02.169589 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03605f6f-48d3-4119-91ea-8664805593e3 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:02.170498 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03605f6f-48d3-4119-91ea-8664805593e3 None admin] RBAC: Adding query filter params (name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:02.170978 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03605f6f-48d3-4119-91ea-8664805593e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:02.172504 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03605f6f-48d3-4119-91ea-8664805593e3 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:02.181479 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 273/552] 10.222.0.23 () {56 vars in 1103 bytes} [Fri May 6 14:45:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:02.187144 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03605f6f-48d3-4119-91ea-8664805593e3 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:02.207632 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a65f3c2-ef29-4481-8337-b3528e11bc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:02.208224 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a65f3c2-ef29-4481-8337-b3528e11bc3c None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:02.208507 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a65f3c2-ef29-4481-8337-b3528e11bc3c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:02.208806 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a65f3c2-ef29-4481-8337-b3528e11bc3c None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:02.210732 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a65f3c2-ef29-4481-8337-b3528e11bc3c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:02.211039 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a65f3c2-ef29-4481-8337-b3528e11bc3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:02.212482 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a65f3c2-ef29-4481-8337-b3528e11bc3c None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:02.243011 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 280/553] 10.222.0.23 () {56 vars in 1135 bytes} [Fri May 6 14:45:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:02.250076 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a65f3c2-ef29-4481-8337-b3528e11bc3c None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:02.299967 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03605f6f-48d3-4119-91ea-8664805593e3 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:02.335409 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b92657-6dc8-4903-8057-b3cfaba7a28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:02.337053 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b92657-6dc8-4903-8057-b3cfaba7a28d None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:02.337542 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b92657-6dc8-4903-8057-b3cfaba7a28d None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:02.338038 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b92657-6dc8-4903-8057-b3cfaba7a28d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:02.351938 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b92657-6dc8-4903-8057-b3cfaba7a28d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:02.353838 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b92657-6dc8-4903-8057-b3cfaba7a28d None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:02.379596 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-84b92657-6dc8-4903-8057-b3cfaba7a28d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:02.379596 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:02.379596 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:02.379596 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:02.379596 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:02.379596 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:02.379596 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:02.379596 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:02.379596 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:02.379596 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:02.379596 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:02.379596 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:02.379596 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:02.379596 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:02.379596 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:02.380568 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:02.380568 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:02.380568 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:02.380568 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:02.380568 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:02.380568 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:02.380568 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:02.380568 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:02.380568 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:02.380568 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:02.380568 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:02.380568 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:02.380568 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:02.380568 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:02.380568 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:02.380568 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:02.380568 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:02.382210 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:02.382210 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:02.382210 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:02.382210 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:02.382451 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a65f3c2-ef29-4481-8337-b3528e11bc3c None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:02.390594 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 274/554] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:45:02.396398 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b92657-6dc8-4903-8057-b3cfaba7a28d None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:02.437412 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4010dc41-15dd-42e1-a0a3-7cc87bdb50fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:02.438131 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010dc41-15dd-42e1-a0a3-7cc87bdb50fa None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:02.438415 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010dc41-15dd-42e1-a0a3-7cc87bdb50fa None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:02.438692 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010dc41-15dd-42e1-a0a3-7cc87bdb50fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:02.525200 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4010dc41-15dd-42e1-a0a3-7cc87bdb50fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:02.526717 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4010dc41-15dd-42e1-a0a3-7cc87bdb50fa None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:02.555863 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b92657-6dc8-4903-8057-b3cfaba7a28d None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:02.597451 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda4bab4-0444-4254-bde1-cc9067755eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:02.598279 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4bab4-0444-4254-bde1-cc9067755eae None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:02.598566 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4bab4-0444-4254-bde1-cc9067755eae None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:02.599696 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4bab4-0444-4254-bde1-cc9067755eae None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:02.600994 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda4bab4-0444-4254-bde1-cc9067755eae None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:02.601310 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda4bab4-0444-4254-bde1-cc9067755eae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:02.602861 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda4bab4-0444-4254-bde1-cc9067755eae None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:02.617723 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 275/555] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:02.619514 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08608385-39a4-45d3-88e2-b698ed8098dd None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:02.619950 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08608385-39a4-45d3-88e2-b698ed8098dd None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:02.620366 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08608385-39a4-45d3-88e2-b698ed8098dd None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:02.622487 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 276/556] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:45:02.624462 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08608385-39a4-45d3-88e2-b698ed8098dd None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:02.634250 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 281/557] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:45:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 06 14:45:02.635593 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647ee92-d6cf-443b-a382-bf3fb145c5bf None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:02.635884 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647ee92-d6cf-443b-a382-bf3fb145c5bf None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:02.636161 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647ee92-d6cf-443b-a382-bf3fb145c5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:02.653071 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-0647ee92-d6cf-443b-a382-bf3fb145c5bf None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:02.654264 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-0647ee92-d6cf-443b-a382-bf3fb145c5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:02.721974 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 282/558] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:45:02.723087 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0647ee92-d6cf-443b-a382-bf3fb145c5bf None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:02.742689 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08608385-39a4-45d3-88e2-b698ed8098dd None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:02.784213 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cab110-293a-49aa-b8d9-e4ba336755f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:02.785072 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cab110-293a-49aa-b8d9-e4ba336755f0 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:02.785857 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cab110-293a-49aa-b8d9-e4ba336755f0 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:02.786134 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cab110-293a-49aa-b8d9-e4ba336755f0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:02.795834 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cab110-293a-49aa-b8d9-e4ba336755f0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:02.797563 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cab110-293a-49aa-b8d9-e4ba336755f0 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:02.816252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-84cab110-293a-49aa-b8d9-e4ba336755f0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 06 14:45:02.816252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:02.816252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:02.816252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:02.816252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:02.816252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:02.816252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:02.816252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:02.816252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:02.816252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:02.816252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:02.816252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:02.816252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 06 14:45:02.816252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_user(user_id) May 06 14:45:02.816252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 06 14:45:02.816252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 06 14:45:02.819128 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:02.819128 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:02.819128 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 06 14:45:02.819128 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:02.819128 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 06 14:45:02.819128 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:02.819128 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:02.819128 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:02.819128 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:02.819128 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:02.819128 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:02.819128 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:02.819128 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:02.819128 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:02.819128 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:02.819128 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:02.819128 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:02.819128 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:02.821248 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:02.821248 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:02.821248 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 06 14:45:02.821248 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 06 14:45:02.821248 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 06 14:45:02.821248 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 06 14:45:02.821248 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 06 14:45:02.821248 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 06 14:45:02.821248 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 06 14:45:02.821248 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:02.821248 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 277/559] 10.222.0.23 () {56 vars in 1093 bytes} [Fri May 6 14:45:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:02.821248 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cab110-293a-49aa-b8d9-e4ba336755f0 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:02.843305 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0647ee92-d6cf-443b-a382-bf3fb145c5bf None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:02.879603 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c7d7b-ca08-4576-926f-5e1937e8042e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:02.881507 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c7d7b-ca08-4576-926f-5e1937e8042e None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:02.882013 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c7d7b-ca08-4576-926f-5e1937e8042e None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:02.882486 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c7d7b-ca08-4576-926f-5e1937e8042e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:02.893304 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c7d7b-ca08-4576-926f-5e1937e8042e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:02.895054 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c7d7b-ca08-4576-926f-5e1937e8042e None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:02.915202 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-593c7d7b-ca08-4576-926f-5e1937e8042e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:02.915202 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:02.915202 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:02.915202 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:02.915202 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:02.915202 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:02.915202 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:02.915202 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:02.915202 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:02.915202 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:02.915202 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:02.915202 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:02.915202 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:02.915202 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:02.915202 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:02.916078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:02.916078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:02.916078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:02.916078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:02.916078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:02.916078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:02.916078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:02.916078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:02.916078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:02.916078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:02.916078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:02.916078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:02.916078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:02.916078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:02.916078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:02.916078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:02.916078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:02.917395 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:02.917395 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:02.917395 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:02.917395 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:02.923751 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 283/560] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:02.924906 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c7d7b-ca08-4576-926f-5e1937e8042e None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:02.926706 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cab110-293a-49aa-b8d9-e4ba336755f0 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:02.961525 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f21c824-1ba4-4958-bebd-4366b81c56ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:02.962321 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f21c824-1ba4-4958-bebd-4366b81c56ca None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:02.962601 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f21c824-1ba4-4958-bebd-4366b81c56ca None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:02.962878 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f21c824-1ba4-4958-bebd-4366b81c56ca None admin] PATH_INFO: `/v3/roles/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:02.973890 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f21c824-1ba4-4958-bebd-4366b81c56ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:45:02.983965 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-5f21c824-1ba4-4958-bebd-4366b81c56ca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 06 14:45:02.983965 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:02.983965 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:02.983965 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:02.983965 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:02.983965 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:02.983965 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:02.983965 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:02.983965 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:02.983965 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:02.983965 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:02.983965 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:02.983965 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:45:02.983965 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:45:02.983965 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:45:02.983965 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:45:02.985037 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:45:02.985037 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:45:02.985037 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:45:02.985037 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:45:02.985037 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:02.985037 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:02.985037 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:02.985037 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:02.985037 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:02.985037 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:02.985037 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:02.985037 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:02.985037 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:02.985037 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:02.985037 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:02.985037 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:02.986345 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:02.986345 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:02.986345 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:02.986345 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:02.986345 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:45:02.986345 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:45:02.986345 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:45:02.986345 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:45:02.986345 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:45:02.986345 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:45:02.986345 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 06 14:45:02.986345 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:02.986345 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 278/561] 10.222.0.23 () {56 vars in 1102 bytes} [Fri May 6 14:45:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:02.987166 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f21c824-1ba4-4958-bebd-4366b81c56ca None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:03.040945 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c7d7b-ca08-4576-926f-5e1937e8042e None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:03.078317 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabacb5b-0655-47cc-874e-245e36618871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:03.079450 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabacb5b-0655-47cc-874e-245e36618871 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:03.079960 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabacb5b-0655-47cc-874e-245e36618871 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:03.080869 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabacb5b-0655-47cc-874e-245e36618871 None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:03.081744 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabacb5b-0655-47cc-874e-245e36618871 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:03.083029 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabacb5b-0655-47cc-874e-245e36618871 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:03.084546 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabacb5b-0655-47cc-874e-245e36618871 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:03.100315 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f21c824-1ba4-4958-bebd-4366b81c56ca None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:03.112062 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 284/562] 10.222.0.23 () {56 vars in 1133 bytes} [Fri May 6 14:45:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:03.113245 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabacb5b-0655-47cc-874e-245e36618871 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:03.139399 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f2c0b-e2fb-49e6-85ee-2fc4b365db57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:03.140116 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f2c0b-e2fb-49e6-85ee-2fc4b365db57 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:03.140403 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f2c0b-e2fb-49e6-85ee-2fc4b365db57 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:03.140723 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f2c0b-e2fb-49e6-85ee-2fc4b365db57 None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:03.141500 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487f2c0b-e2fb-49e6-85ee-2fc4b365db57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:03.141807 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487f2c0b-e2fb-49e6-85ee-2fc4b365db57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:03.143222 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487f2c0b-e2fb-49e6-85ee-2fc4b365db57 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:03.161384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 279/563] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:03.162891 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f2c0b-e2fb-49e6-85ee-2fc4b365db57 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:03.231908 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabacb5b-0655-47cc-874e-245e36618871 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:03.264517 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f6f80c-9362-47f9-8ee2-23a8e6c76e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:03.265221 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f6f80c-9362-47f9-8ee2-23a8e6c76e03 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:03.265508 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f6f80c-9362-47f9-8ee2-23a8e6c76e03 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:03.265792 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f6f80c-9362-47f9-8ee2-23a8e6c76e03 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:03.266532 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f6f80c-9362-47f9-8ee2-23a8e6c76e03 None admin] RBAC: Adding query filter params (name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:03.266838 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f6f80c-9362-47f9-8ee2-23a8e6c76e03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:03.267908 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f2c0b-e2fb-49e6-85ee-2fc4b365db57 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:03.268200 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f6f80c-9362-47f9-8ee2-23a8e6c76e03 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:03.277174 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 285/564] 10.222.0.23 () {56 vars in 1103 bytes} [Fri May 6 14:45:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:03.281779 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f6f80c-9362-47f9-8ee2-23a8e6c76e03 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:03.304302 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64df4c0b-c8b1-420d-b5e9-e7d480d70c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:03.305029 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df4c0b-c8b1-420d-b5e9-e7d480d70c6e None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:03.305311 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df4c0b-c8b1-420d-b5e9-e7d480d70c6e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:03.305588 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df4c0b-c8b1-420d-b5e9-e7d480d70c6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:03.315620 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64df4c0b-c8b1-420d-b5e9-e7d480d70c6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:03.317123 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64df4c0b-c8b1-420d-b5e9-e7d480d70c6e None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:03.336355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-64df4c0b-c8b1-420d-b5e9-e7d480d70c6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:03.336355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:03.336355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:03.336355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:03.336355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:03.336355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:03.336355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:03.336355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:03.336355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:03.336355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:03.336355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:03.336355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:03.336355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:03.336355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:03.336355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:03.342379 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:03.342379 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:03.342379 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:03.342379 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:03.342379 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:03.342379 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:03.342379 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:03.342379 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:03.342379 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:03.342379 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:03.342379 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:03.342379 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:03.342379 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:03.342379 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:03.342379 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:03.342379 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:03.342379 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:03.343369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:03.343369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:03.343369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:03.343369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:03.343369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 280/565] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:03.343369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300cc7bf-3e51-4977-8b2a-3fa0bbc8b2c8 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:03.343369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300cc7bf-3e51-4977-8b2a-3fa0bbc8b2c8 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:03.343369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300cc7bf-3e51-4977-8b2a-3fa0bbc8b2c8 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:03.343369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 281/566] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:45:03.343858 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300cc7bf-3e51-4977-8b2a-3fa0bbc8b2c8 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:03.387523 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f6f80c-9362-47f9-8ee2-23a8e6c76e03 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:03.425782 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a481897-2bb3-4244-882e-c73d620d539d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:03.426906 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a481897-2bb3-4244-882e-c73d620d539d None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:03.427350 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a481897-2bb3-4244-882e-c73d620d539d None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:03.427810 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a481897-2bb3-4244-882e-c73d620d539d None admin] PATH_INFO: `/v3/projects/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:03.435364 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a481897-2bb3-4244-882e-c73d620d539d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:03.436818 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a481897-2bb3-4244-882e-c73d620d539d None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:03.448914 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300cc7bf-3e51-4977-8b2a-3fa0bbc8b2c8 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:03.451549 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-5a481897-2bb3-4244-882e-c73d620d539d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:03.451549 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:03.451549 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:03.451549 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:03.451549 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:03.451549 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:03.451549 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:03.451549 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:03.451549 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:03.451549 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:03.451549 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:03.451549 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:03.451549 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 06 14:45:03.451549 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(project_id) May 06 14:45:03.451549 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 06 14:45:03.452635 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 06 14:45:03.452635 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:03.452635 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:03.452635 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:03.452635 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:03.452635 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:03.452635 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:03.452635 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:03.452635 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:03.452635 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:03.452635 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:03.452635 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:03.452635 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:03.452635 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:03.452635 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:03.452635 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:03.452635 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:03.453549 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 06 14:45:03.453549 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 06 14:45:03.453549 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 06 14:45:03.453549 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 06 14:45:03.453549 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 06 14:45:03.453549 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 06 14:45:03.453549 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:03.453549 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:03.453549 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 286/567] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:45:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:03.453549 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a481897-2bb3-4244-882e-c73d620d539d None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:03.490215 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb2d1e8-bada-44d9-9a91-b50f3cf9504f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:03.491343 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2d1e8-bada-44d9-9a91-b50f3cf9504f None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:03.491624 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2d1e8-bada-44d9-9a91-b50f3cf9504f None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:03.491894 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2d1e8-bada-44d9-9a91-b50f3cf9504f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:03.501642 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb2d1e8-bada-44d9-9a91-b50f3cf9504f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:03.503115 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb2d1e8-bada-44d9-9a91-b50f3cf9504f None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:03.521113 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-dfb2d1e8-bada-44d9-9a91-b50f3cf9504f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:03.521113 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:03.521113 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:03.521113 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:03.521113 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:03.521113 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:03.521113 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:03.521113 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:03.521113 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:03.521113 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:03.521113 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:03.521113 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:03.521113 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:03.521113 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:03.521113 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:03.522695 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:03.522695 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:03.522695 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:03.522695 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:03.522695 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:03.522695 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:03.522695 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:03.522695 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:03.522695 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:03.522695 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:03.522695 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:03.522695 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:03.522695 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:03.522695 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:03.522695 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:03.522695 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:03.522695 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:03.523576 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:03.523576 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:03.523576 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:03.523576 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:03.523576 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 282/568] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:03.524707 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806efc23-e492-4dbb-a76e-54a5b1c6307f None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:03.524953 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806efc23-e492-4dbb-a76e-54a5b1c6307f None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:03.525231 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806efc23-e492-4dbb-a76e-54a5b1c6307f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:03.542355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-806efc23-e492-4dbb-a76e-54a5b1c6307f None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:03.543527 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-806efc23-e492-4dbb-a76e-54a5b1c6307f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:03.563334 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a481897-2bb3-4244-882e-c73d620d539d None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:03.595501 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf73ae63-914b-468e-8ffa-fa0e7adeb9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:03.596189 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf73ae63-914b-468e-8ffa-fa0e7adeb9b7 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:03.596473 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf73ae63-914b-468e-8ffa-fa0e7adeb9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:03.596778 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf73ae63-914b-468e-8ffa-fa0e7adeb9b7 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:03.597488 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf73ae63-914b-468e-8ffa-fa0e7adeb9b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:03.597792 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf73ae63-914b-468e-8ffa-fa0e7adeb9b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:03.599147 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf73ae63-914b-468e-8ffa-fa0e7adeb9b7 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:03.607559 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 283/569] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:45:03.608383 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806efc23-e492-4dbb-a76e-54a5b1c6307f None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:03.613420 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 287/570] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:03.613420 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf73ae63-914b-468e-8ffa-fa0e7adeb9b7 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:03.704551 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806efc23-e492-4dbb-a76e-54a5b1c6307f None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:03.706830 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf73ae63-914b-468e-8ffa-fa0e7adeb9b7 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:03.736374 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2218b7c6-2f1d-4a11-9665-11ff5c9b60de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:03.737058 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218b7c6-2f1d-4a11-9665-11ff5c9b60de None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:03.737337 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218b7c6-2f1d-4a11-9665-11ff5c9b60de None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:03.737610 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218b7c6-2f1d-4a11-9665-11ff5c9b60de None admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:03.738098 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd898fb-c0c5-4eef-99bf-769d6c192154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:03.738377 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2218b7c6-2f1d-4a11-9665-11ff5c9b60de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:03.738680 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2218b7c6-2f1d-4a11-9665-11ff5c9b60de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:03.738772 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd898fb-c0c5-4eef-99bf-769d6c192154 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:03.739019 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd898fb-c0c5-4eef-99bf-769d6c192154 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:03.739293 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd898fb-c0c5-4eef-99bf-769d6c192154 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:03.739995 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd898fb-c0c5-4eef-99bf-769d6c192154 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:03.740084 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2218b7c6-2f1d-4a11-9665-11ff5c9b60de None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:03.740922 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd898fb-c0c5-4eef-99bf-769d6c192154 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:03.742445 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd898fb-c0c5-4eef-99bf-769d6c192154 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:03.752109 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 284/571] 10.222.0.23 () {56 vars in 1148 bytes} [Fri May 6 14:45:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:03.753372 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9eb4c-f688-4e92-9e47-ab75bea4196c None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:03.753875 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 288/572] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:03.756708 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd898fb-c0c5-4eef-99bf-769d6c192154 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:03.756867 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9eb4c-f688-4e92-9e47-ab75bea4196c None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:03.756867 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9eb4c-f688-4e92-9e47-ab75bea4196c None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:03.756867 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 285/573] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:45:03.757763 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e9eb4c-f688-4e92-9e47-ab75bea4196c None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:03.852132 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd898fb-c0c5-4eef-99bf-769d6c192154 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:03.856050 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e9eb4c-f688-4e92-9e47-ab75bea4196c None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:03.884521 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bf944f-324e-42e8-9a0e-039ba9647bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:03.885374 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf944f-324e-42e8-9a0e-039ba9647bbd None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:03.885653 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf944f-324e-42e8-9a0e-039ba9647bbd None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:03.885922 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf944f-324e-42e8-9a0e-039ba9647bbd None admin] PATH_INFO: `/v3/projects/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:03.888225 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8788562b-a818-4f6c-9eaa-98839450a0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:03.889031 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8788562b-a818-4f6c-9eaa-98839450a0de None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:03.889314 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8788562b-a818-4f6c-9eaa-98839450a0de None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:03.889593 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8788562b-a818-4f6c-9eaa-98839450a0de None admin] PATH_INFO: `/v3/roles/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:03.892297 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bf944f-324e-42e8-9a0e-039ba9647bbd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:03.893763 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bf944f-324e-42e8-9a0e-039ba9647bbd None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:03.900136 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8788562b-a818-4f6c-9eaa-98839450a0de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:45:03.908038 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-82bf944f-324e-42e8-9a0e-039ba9647bbd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:03.908038 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:03.908038 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:03.908038 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:03.908038 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:03.908038 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:03.908038 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:03.908038 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:03.908038 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:03.908038 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:03.908038 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:03.908038 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:03.908038 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 06 14:45:03.908038 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(project_id) May 06 14:45:03.908038 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 06 14:45:03.909093 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 06 14:45:03.909093 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:03.909093 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:03.909093 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:03.909093 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:03.909093 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:03.909093 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:03.909093 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:03.909093 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:03.909093 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:03.909093 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:03.909093 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:03.909093 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:03.909093 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:03.909093 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:03.909093 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:03.909093 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:03.915524 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 06 14:45:03.915524 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 06 14:45:03.915524 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 06 14:45:03.915524 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 06 14:45:03.915524 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 06 14:45:03.915524 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 06 14:45:03.915524 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:03.915524 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:03.915964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-8788562b-a818-4f6c-9eaa-98839450a0de None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 06 14:45:03.915964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:03.915964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:03.915964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:03.915964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:03.915964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:03.915964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:03.915964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:03.915964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:03.915964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:03.915964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:03.915964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:03.915964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:45:03.915964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:45:03.915964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:45:03.915964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:45:03.916887 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:45:03.916887 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:45:03.916887 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:45:03.916887 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:45:03.916887 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:03.916887 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:03.916887 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:03.916887 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:03.916887 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:03.916887 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:03.916887 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:03.916887 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:03.916887 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:03.916887 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:03.916887 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:03.916887 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:03.917995 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:03.917995 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:03.917995 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:03.917995 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:03.917995 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:45:03.917995 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:45:03.917995 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:45:03.917995 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:45:03.917995 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:45:03.917995 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:45:03.917995 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 06 14:45:03.917995 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:03.918765 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 289/574] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:45:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:03.918765 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bf944f-324e-42e8-9a0e-039ba9647bbd None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:03.918911 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 286/575] 10.222.0.23 () {56 vars in 1102 bytes} [Fri May 6 14:45:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:03.918911 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8788562b-a818-4f6c-9eaa-98839450a0de None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:04.007666 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8788562b-a818-4f6c-9eaa-98839450a0de None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:04.007878 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bf944f-324e-42e8-9a0e-039ba9647bbd None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:04.042322 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daeee93-b54e-460a-9b26-63194fa38924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:04.042883 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67fab3f-5c97-41a2-81f5-e7ffcf097d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:04.043118 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daeee93-b54e-460a-9b26-63194fa38924 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:04.043398 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daeee93-b54e-460a-9b26-63194fa38924 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:04.043540 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67fab3f-5c97-41a2-81f5-e7ffcf097d5b None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:04.043698 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daeee93-b54e-460a-9b26-63194fa38924 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:04.043848 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67fab3f-5c97-41a2-81f5-e7ffcf097d5b None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:04.044067 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67fab3f-5c97-41a2-81f5-e7ffcf097d5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:04.044919 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67fab3f-5c97-41a2-81f5-e7ffcf097d5b None admin] RBAC: Adding query filter params (role.id=f5641817ee56438496afc6135d37a4c0, scope.project.id=d5398feee6eb490cb49bb261448a1a9b, user.id=bf9eed62c8f1409282c2af6488426a0f) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:04.045225 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67fab3f-5c97-41a2-81f5-e7ffcf097d5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:04.046650 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67fab3f-5c97-41a2-81f5-e7ffcf097d5b None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:04.051496 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daeee93-b54e-460a-9b26-63194fa38924 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:04.052292 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 290/576] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:45:03 2022] GET /identity/v3/role_assignments?user.id=bf9eed62c8f1409282c2af6488426a0f&scope.project.id=d5398feee6eb490cb49bb261448a1a9b&role.id=f5641817ee56438496afc6135d37a4c0 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:04.053059 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daeee93-b54e-460a-9b26-63194fa38924 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:04.054172 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521bc15-2f67-4c11-81fd-1850a8577f43 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:04.054455 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521bc15-2f67-4c11-81fd-1850a8577f43 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:04.054728 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521bc15-2f67-4c11-81fd-1850a8577f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:04.068543 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-0daeee93-b54e-460a-9b26-63194fa38924 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 06 14:45:04.068543 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:04.068543 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:04.068543 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:04.068543 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:04.068543 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:04.068543 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:04.068543 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:04.068543 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:04.068543 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:04.068543 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:04.068543 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:04.068543 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 06 14:45:04.068543 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_user(user_id) May 06 14:45:04.068543 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 06 14:45:04.068543 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 06 14:45:04.069846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:04.069846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:04.069846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 06 14:45:04.069846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:04.069846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 06 14:45:04.069846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:04.069846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:04.069846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:04.069846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:04.069846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:04.069846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:04.069846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:04.069846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:04.069846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:04.069846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:04.069846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:04.069846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:04.069846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:04.071083 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:04.071083 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:04.071083 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 06 14:45:04.071083 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 06 14:45:04.071083 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 06 14:45:04.071083 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 06 14:45:04.071083 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 06 14:45:04.071083 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 06 14:45:04.071083 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 06 14:45:04.071083 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:04.072151 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 287/577] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:45:03 2022] GET /identity/v3/users/cinder => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:04.072764 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daeee93-b54e-460a-9b26-63194fa38924 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:04.077466 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-c521bc15-2f67-4c11-81fd-1850a8577f43 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:04.078787 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-c521bc15-2f67-4c11-81fd-1850a8577f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:04.167120 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 291/578] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:45:04.168011 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c521bc15-2f67-4c11-81fd-1850a8577f43 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:04.194344 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daeee93-b54e-460a-9b26-63194fa38924 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:04.233962 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e71a5c-6858-440b-9417-8235292a4d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:04.234640 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e71a5c-6858-440b-9417-8235292a4d5f None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:04.234923 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e71a5c-6858-440b-9417-8235292a4d5f None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:04.235196 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e71a5c-6858-440b-9417-8235292a4d5f None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:04.235942 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e71a5c-6858-440b-9417-8235292a4d5f None admin] RBAC: Adding query filter params (name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:04.236672 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e71a5c-6858-440b-9417-8235292a4d5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:04.237641 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e71a5c-6858-440b-9417-8235292a4d5f None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:04.247398 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 288/579] 10.222.0.23 () {56 vars in 1103 bytes} [Fri May 6 14:45:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:04.248078 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e71a5c-6858-440b-9417-8235292a4d5f None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:04.273138 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c521bc15-2f67-4c11-81fd-1850a8577f43 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:04.308337 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1057a304-0574-4ab4-b3ae-75040dd9febb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:04.309043 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057a304-0574-4ab4-b3ae-75040dd9febb None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:04.309331 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057a304-0574-4ab4-b3ae-75040dd9febb None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:04.309608 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057a304-0574-4ab4-b3ae-75040dd9febb None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:04.310409 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1057a304-0574-4ab4-b3ae-75040dd9febb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:04.310722 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1057a304-0574-4ab4-b3ae-75040dd9febb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:04.312166 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1057a304-0574-4ab4-b3ae-75040dd9febb None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:04.324569 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 292/580] 10.222.0.23 () {56 vars in 1148 bytes} [Fri May 6 14:45:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:04.325460 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1057a304-0574-4ab4-b3ae-75040dd9febb None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:04.374819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e71a5c-6858-440b-9417-8235292a4d5f None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:04.411824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045de7db-0f8f-4fbd-ba5a-917013582ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:04.412647 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045de7db-0f8f-4fbd-ba5a-917013582ee0 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:04.413030 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045de7db-0f8f-4fbd-ba5a-917013582ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:04.413400 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045de7db-0f8f-4fbd-ba5a-917013582ee0 None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:04.414321 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045de7db-0f8f-4fbd-ba5a-917013582ee0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:04.414725 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045de7db-0f8f-4fbd-ba5a-917013582ee0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:04.416227 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045de7db-0f8f-4fbd-ba5a-917013582ee0 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:04.426641 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1057a304-0574-4ab4-b3ae-75040dd9febb None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:04.455026 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 289/581] 10.222.0.23 () {56 vars in 1137 bytes} [Fri May 6 14:45:04 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:04.455026 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045de7db-0f8f-4fbd-ba5a-917013582ee0 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:04.465073 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc9b404-6590-4a39-898f-aff84a394ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:04.465892 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9b404-6590-4a39-898f-aff84a394ef5 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:04.466178 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9b404-6590-4a39-898f-aff84a394ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:04.466455 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9b404-6590-4a39-898f-aff84a394ef5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:04.476557 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc9b404-6590-4a39-898f-aff84a394ef5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:45:04.484195 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-3cc9b404-6590-4a39-898f-aff84a394ef5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 06 14:45:04.484195 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:04.484195 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:04.484195 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:04.484195 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:04.484195 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:04.484195 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:04.484195 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:04.484195 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:04.484195 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:04.484195 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:04.484195 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:04.484195 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:45:04.484195 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:45:04.484195 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:45:04.484195 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:45:04.485364 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:45:04.485364 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:45:04.485364 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:45:04.485364 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:45:04.485364 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:04.485364 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:04.485364 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:04.485364 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:04.485364 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:04.485364 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:04.485364 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:04.485364 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:04.485364 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:04.485364 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:04.485364 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:04.485364 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:04.486383 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:04.486383 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:04.486383 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:04.486383 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:04.486383 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:45:04.486383 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:45:04.486383 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:45:04.486383 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:45:04.486383 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:45:04.486383 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:45:04.486383 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 06 14:45:04.486383 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:04.490786 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 293/582] 10.222.0.23 () {56 vars in 1102 bytes} [Fri May 6 14:45:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:04.491446 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc9b404-6590-4a39-898f-aff84a394ef5 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:04.562464 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045de7db-0f8f-4fbd-ba5a-917013582ee0 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:04.599118 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1aa805-ba23-42bd-bbd0-5c8c7f50b59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:04.600044 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1aa805-ba23-42bd-bbd0-5c8c7f50b59a None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:04.600452 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1aa805-ba23-42bd-bbd0-5c8c7f50b59a None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:04.600892 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1aa805-ba23-42bd-bbd0-5c8c7f50b59a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:04.601959 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc9b404-6590-4a39-898f-aff84a394ef5 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:04.611561 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1aa805-ba23-42bd-bbd0-5c8c7f50b59a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:04.613065 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1aa805-ba23-42bd-bbd0-5c8c7f50b59a None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:04.632494 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-0f1aa805-ba23-42bd-bbd0-5c8c7f50b59a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:04.632494 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:04.632494 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:04.632494 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:04.632494 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:04.632494 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:04.632494 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:04.632494 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:04.632494 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:04.632494 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:04.632494 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:04.632494 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:04.632494 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:04.632494 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:04.632494 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:04.633630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:04.633630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:04.633630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:04.633630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:04.633630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:04.633630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:04.633630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:04.633630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:04.633630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:04.633630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:04.633630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:04.633630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:04.633630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:04.633630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:04.633630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:04.633630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:04.633630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:04.634646 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:04.634646 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:04.634646 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:04.634646 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:04.638194 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 290/583] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:04.638194 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1aa805-ba23-42bd-bbd0-5c8c7f50b59a None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:04.650800 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a5288f-706a-4add-a2a6-4c0ed1aa40e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:04.651795 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5288f-706a-4add-a2a6-4c0ed1aa40e3 None admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:04.652151 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5288f-706a-4add-a2a6-4c0ed1aa40e3 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:04.652497 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5288f-706a-4add-a2a6-4c0ed1aa40e3 None admin] PATH_INFO: `/v3/projects/d5398feee6eb490cb49bb261448a1a9b/users/fc8cf09642ec4e10a80badcf8b585ef8/roles/f5641817ee56438496afc6135d37a4c0` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:04.668964 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a5288f-706a-4add-a2a6-4c0ed1aa40e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=d5398feee6eb490cb49bb261448a1a9b, user_id=fc8cf09642ec4e10a80badcf8b585ef8, role_id=f5641817ee56438496afc6135d37a4c0)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:04.670432 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a5288f-706a-4add-a2a6-4c0ed1aa40e3 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:04.688801 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 294/584] 10.222.0.23 () {58 vars in 1439 bytes} [Fri May 6 14:45:04 2022] PUT /identity/v3/projects/d5398feee6eb490cb49bb261448a1a9b/users/fc8cf09642ec4e10a80badcf8b585ef8/roles/f5641817ee56438496afc6135d37a4c0 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:45:04.688801 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a5288f-706a-4add-a2a6-4c0ed1aa40e3 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:04.769219 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1aa805-ba23-42bd-bbd0-5c8c7f50b59a None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:04.806415 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def49638-e68f-4eff-9dd6-72f0c902f58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:04.807329 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def49638-e68f-4eff-9dd6-72f0c902f58d None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:04.807719 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def49638-e68f-4eff-9dd6-72f0c902f58d None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:04.808094 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def49638-e68f-4eff-9dd6-72f0c902f58d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:04.810496 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a5288f-706a-4add-a2a6-4c0ed1aa40e3 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:04.820740 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def49638-e68f-4eff-9dd6-72f0c902f58d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:04.822436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def49638-e68f-4eff-9dd6-72f0c902f58d None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:04.841628 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-def49638-e68f-4eff-9dd6-72f0c902f58d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:04.841628 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:04.841628 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:04.841628 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:04.841628 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:04.841628 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:04.841628 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:04.841628 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:04.841628 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:04.841628 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:04.841628 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:04.841628 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:04.841628 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:04.841628 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:04.841628 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:04.842755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:04.842755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:04.842755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:04.842755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:04.842755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:04.842755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:04.842755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:04.842755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:04.842755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:04.842755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:04.842755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:04.842755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:04.842755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:04.842755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:04.842755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:04.842755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:04.842755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:04.843855 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:04.843855 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:04.843855 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:04.843855 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:04.845320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 291/585] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:04.846288 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def49638-e68f-4eff-9dd6-72f0c902f58d None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:04.854502 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355cf422-85f3-47db-b515-e8cbcadcda56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:04.855185 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355cf422-85f3-47db-b515-e8cbcadcda56 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:04.855469 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355cf422-85f3-47db-b515-e8cbcadcda56 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:04.855813 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355cf422-85f3-47db-b515-e8cbcadcda56 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:04.856696 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355cf422-85f3-47db-b515-e8cbcadcda56 None admin] RBAC: Adding query filter params (name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:04.857003 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355cf422-85f3-47db-b515-e8cbcadcda56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:04.858380 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355cf422-85f3-47db-b515-e8cbcadcda56 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:04.873060 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 295/586] 10.222.0.23 () {56 vars in 1103 bytes} [Fri May 6 14:45:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:04.874040 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355cf422-85f3-47db-b515-e8cbcadcda56 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:04.960852 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def49638-e68f-4eff-9dd6-72f0c902f58d None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:04.993517 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8acf54-3b45-45d7-a665-91143d37bd9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:04.994329 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8acf54-3b45-45d7-a665-91143d37bd9f None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:04.994890 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8acf54-3b45-45d7-a665-91143d37bd9f None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:04.995096 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8acf54-3b45-45d7-a665-91143d37bd9f None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:04.995919 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8acf54-3b45-45d7-a665-91143d37bd9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:04.996321 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8acf54-3b45-45d7-a665-91143d37bd9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:04.997836 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8acf54-3b45-45d7-a665-91143d37bd9f None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:04.998329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355cf422-85f3-47db-b515-e8cbcadcda56 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:05.010750 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 292/587] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:05.011741 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8acf54-3b45-45d7-a665-91143d37bd9f None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:05.039928 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e8818-125c-42f3-af64-14526b55c793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:05.041026 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e8818-125c-42f3-af64-14526b55c793 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:05.041463 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e8818-125c-42f3-af64-14526b55c793 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:05.042178 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e8818-125c-42f3-af64-14526b55c793 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:05.043327 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467e8818-125c-42f3-af64-14526b55c793 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:05.043805 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467e8818-125c-42f3-af64-14526b55c793 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:05.046038 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467e8818-125c-42f3-af64-14526b55c793 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:05.061614 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 296/588] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:05.062749 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e8818-125c-42f3-af64-14526b55c793 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:05.122155 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8acf54-3b45-45d7-a665-91143d37bd9f None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:05.168223 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ed044-30d3-4712-8f9c-06f34bb89155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:05.170444 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ed044-30d3-4712-8f9c-06f34bb89155 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:05.171028 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ed044-30d3-4712-8f9c-06f34bb89155 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:05.171453 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ed044-30d3-4712-8f9c-06f34bb89155 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:05.177444 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e8818-125c-42f3-af64-14526b55c793 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:05.182913 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22ed044-30d3-4712-8f9c-06f34bb89155 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:05.184621 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22ed044-30d3-4712-8f9c-06f34bb89155 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:05.203819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-c22ed044-30d3-4712-8f9c-06f34bb89155 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:05.203819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:05.203819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:05.203819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:05.203819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:05.203819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:05.203819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:05.203819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:05.203819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:05.203819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:05.203819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:05.203819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:05.203819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:05.203819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:05.203819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:05.204840 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:05.204840 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:05.204840 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:05.204840 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:05.204840 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:05.204840 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:05.204840 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:05.204840 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:05.204840 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:05.204840 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:05.204840 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:05.204840 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:05.204840 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:05.204840 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:05.204840 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:05.204840 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:05.204840 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:05.205718 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:05.205718 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:05.205718 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:05.205718 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:05.208132 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 293/589] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:05.210221 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ed044-30d3-4712-8f9c-06f34bb89155 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:05.216100 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f32e7f0-362a-45a1-b668-6305636605fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:05.216407 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32e7f0-362a-45a1-b668-6305636605fc None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:05.216585 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32e7f0-362a-45a1-b668-6305636605fc None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:05.216889 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32e7f0-362a-45a1-b668-6305636605fc None admin] PATH_INFO: `/v3/users/glance` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:05.224632 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f32e7f0-362a-45a1-b668-6305636605fc None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:05.226100 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f32e7f0-362a-45a1-b668-6305636605fc None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:05.241517 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-1f32e7f0-362a-45a1-b668-6305636605fc None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 06 14:45:05.241517 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:05.241517 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:05.241517 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:05.241517 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:05.241517 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:05.241517 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:05.241517 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:05.241517 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:05.241517 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:05.241517 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:05.241517 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:05.241517 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 06 14:45:05.241517 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_user(user_id) May 06 14:45:05.241517 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 06 14:45:05.241517 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 06 14:45:05.242489 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:05.242489 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:05.242489 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 06 14:45:05.242489 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:05.242489 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 06 14:45:05.242489 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:05.242489 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:05.242489 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:05.242489 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:05.242489 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:05.242489 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:05.242489 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:05.242489 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:05.242489 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:05.242489 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:05.242489 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:05.242489 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:05.242489 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:05.243414 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:05.243414 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:05.243414 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 06 14:45:05.243414 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 06 14:45:05.243414 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 06 14:45:05.243414 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 06 14:45:05.243414 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 06 14:45:05.243414 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 06 14:45:05.243414 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 06 14:45:05.243414 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:05.247551 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 297/590] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:45:05 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:05.248326 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f32e7f0-362a-45a1-b668-6305636605fc None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:05.319449 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ed044-30d3-4712-8f9c-06f34bb89155 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:05.351342 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f32e7f0-362a-45a1-b668-6305636605fc None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:05.355643 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7f7f8c-a62e-4784-9e42-d51abdfdc98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:05.356842 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7f7f8c-a62e-4784-9e42-d51abdfdc98c None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:05.357374 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7f7f8c-a62e-4784-9e42-d51abdfdc98c None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:05.357863 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7f7f8c-a62e-4784-9e42-d51abdfdc98c None admin] PATH_INFO: `/v3/projects/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:05.365822 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7f7f8c-a62e-4784-9e42-d51abdfdc98c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:05.367639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7f7f8c-a62e-4784-9e42-d51abdfdc98c None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:05.382505 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-6d7f7f8c-a62e-4784-9e42-d51abdfdc98c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:05.382505 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:05.382505 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:05.382505 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:05.382505 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:05.382505 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:05.382505 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:05.382505 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:05.382505 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:05.382505 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:05.382505 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:05.382505 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:05.382505 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 06 14:45:05.382505 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(project_id) May 06 14:45:05.382505 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 06 14:45:05.383797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 06 14:45:05.383797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:05.383797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:05.383797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:05.383797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:05.383797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:05.383797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:05.383797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:05.383797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:05.383797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:05.383797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:05.383797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:05.383797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:05.383797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:05.383797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:05.383797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:05.383797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:05.384727 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 06 14:45:05.384727 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 06 14:45:05.384727 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 06 14:45:05.384727 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 06 14:45:05.384727 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 06 14:45:05.384727 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 06 14:45:05.384727 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:05.384727 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:05.385923 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 294/591] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:45:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:05.386711 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7f7f8c-a62e-4784-9e42-d51abdfdc98c None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:05.390723 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b106b8df-36c4-4bda-87b4-51ac4712e143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:05.391394 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b106b8df-36c4-4bda-87b4-51ac4712e143 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:05.391677 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b106b8df-36c4-4bda-87b4-51ac4712e143 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:05.391953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b106b8df-36c4-4bda-87b4-51ac4712e143 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:05.392702 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b106b8df-36c4-4bda-87b4-51ac4712e143 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:05.393009 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b106b8df-36c4-4bda-87b4-51ac4712e143 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:05.394408 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b106b8df-36c4-4bda-87b4-51ac4712e143 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:05.406778 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 298/592] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:05.407680 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b106b8df-36c4-4bda-87b4-51ac4712e143 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:05.493182 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7f7f8c-a62e-4784-9e42-d51abdfdc98c None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:05.508436 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b106b8df-36c4-4bda-87b4-51ac4712e143 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:05.529812 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0659deba-8f78-4006-bd01-3a9cc10c1aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:05.531166 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659deba-8f78-4006-bd01-3a9cc10c1aa9 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:05.531656 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659deba-8f78-4006-bd01-3a9cc10c1aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:05.532173 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659deba-8f78-4006-bd01-3a9cc10c1aa9 None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:05.533225 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0659deba-8f78-4006-bd01-3a9cc10c1aa9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:05.533748 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0659deba-8f78-4006-bd01-3a9cc10c1aa9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:05.535451 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0659deba-8f78-4006-bd01-3a9cc10c1aa9 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:05.543342 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc64b3-4be4-4ba9-b99a-a6678b9fea38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:05.544022 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc64b3-4be4-4ba9-b99a-a6678b9fea38 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:05.544293 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc64b3-4be4-4ba9-b99a-a6678b9fea38 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:05.544572 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc64b3-4be4-4ba9-b99a-a6678b9fea38 None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:05.545379 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cc64b3-4be4-4ba9-b99a-a6678b9fea38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:05.545739 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cc64b3-4be4-4ba9-b99a-a6678b9fea38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:05.547099 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cc64b3-4be4-4ba9-b99a-a6678b9fea38 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:05.557554 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 295/593] 10.222.0.23 () {56 vars in 1137 bytes} [Fri May 6 14:45:05 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:05.558743 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0659deba-8f78-4006-bd01-3a9cc10c1aa9 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:05.564117 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 299/594] 10.222.0.23 () {56 vars in 1148 bytes} [Fri May 6 14:45:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:05.565008 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc64b3-4be4-4ba9-b99a-a6678b9fea38 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:05.661419 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0659deba-8f78-4006-bd01-3a9cc10c1aa9 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:05.665318 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc64b3-4be4-4ba9-b99a-a6678b9fea38 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:05.697394 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b23cd-fdd0-47c6-98d6-37dd7ea5e7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:05.698862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b23cd-fdd0-47c6-98d6-37dd7ea5e7af None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:05.699409 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b23cd-fdd0-47c6-98d6-37dd7ea5e7af None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:05.699860 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b23cd-fdd0-47c6-98d6-37dd7ea5e7af None admin] PATH_INFO: `/v3/users/neutron` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:05.699965 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f9986-4be7-4b53-b4c4-996425960e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:05.700858 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f9986-4be7-4b53-b4c4-996425960e77 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:05.701194 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f9986-4be7-4b53-b4c4-996425960e77 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:05.701545 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f9986-4be7-4b53-b4c4-996425960e77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:05.708188 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5b23cd-fdd0-47c6-98d6-37dd7ea5e7af None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:05.710078 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5b23cd-fdd0-47c6-98d6-37dd7ea5e7af None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:05.712013 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8f9986-4be7-4b53-b4c4-996425960e77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:05.713537 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8f9986-4be7-4b53-b4c4-996425960e77 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:05.726126 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-0c5b23cd-fdd0-47c6-98d6-37dd7ea5e7af None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 06 14:45:05.726126 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:05.726126 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:05.726126 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:05.726126 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:05.726126 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:05.726126 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:05.726126 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:05.726126 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:05.726126 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:05.726126 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:05.726126 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:05.726126 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 06 14:45:05.726126 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_user(user_id) May 06 14:45:05.726126 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 06 14:45:05.726126 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 06 14:45:05.727074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:05.727074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:05.727074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 06 14:45:05.727074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:05.727074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 06 14:45:05.727074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:05.727074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:05.727074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:05.727074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:05.727074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:05.727074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:05.727074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:05.727074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:05.727074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:05.727074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:05.727074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:05.727074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:05.727074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:05.728064 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:05.728064 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:05.728064 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 06 14:45:05.728064 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 06 14:45:05.728064 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 06 14:45:05.728064 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 06 14:45:05.728064 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 06 14:45:05.728064 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 06 14:45:05.728064 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 06 14:45:05.728064 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:05.730751 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 296/595] 10.222.0.23 () {56 vars in 1102 bytes} [Fri May 6 14:45:05 2022] GET /identity/v3/users/neutron => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:05.731692 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b23cd-fdd0-47c6-98d6-37dd7ea5e7af None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:05.732318 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-4e8f9986-4be7-4b53-b4c4-996425960e77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:05.732318 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:05.732318 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:05.732318 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:05.732318 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:05.732318 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:05.732318 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:05.732318 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:05.732318 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:05.732318 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:05.732318 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:05.732318 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:05.732318 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:05.732318 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:05.732318 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:05.733233 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:05.733233 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:05.733233 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:05.733233 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:05.733233 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:05.733233 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:05.733233 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:05.733233 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:05.733233 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:05.733233 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:05.733233 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:05.733233 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:05.733233 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:05.733233 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:05.733233 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:05.733233 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:05.733233 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:05.734127 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:05.734127 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:05.734127 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:05.734127 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:05.738419 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 300/596] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:05.739029 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f9986-4be7-4b53-b4c4-996425960e77 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:05.836096 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b23cd-fdd0-47c6-98d6-37dd7ea5e7af None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:05.839268 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f9986-4be7-4b53-b4c4-996425960e77 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:05.872585 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea87666-2d54-4f81-b972-0ffbd773db6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:05.874110 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea87666-2d54-4f81-b972-0ffbd773db6e None admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:05.874630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea87666-2d54-4f81-b972-0ffbd773db6e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:05.874736 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf38899-9c06-40ec-a969-31aab343fb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:05.875274 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea87666-2d54-4f81-b972-0ffbd773db6e None admin] PATH_INFO: `/v3/projects/d5398feee6eb490cb49bb261448a1a9b/users/6409b68b4832421ba590d7bc53131f7c/roles/f5641817ee56438496afc6135d37a4c0` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:05.875585 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf38899-9c06-40ec-a969-31aab343fb55 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:05.875968 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf38899-9c06-40ec-a969-31aab343fb55 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:05.876344 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf38899-9c06-40ec-a969-31aab343fb55 None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:05.877369 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf38899-9c06-40ec-a969-31aab343fb55 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:05.877779 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf38899-9c06-40ec-a969-31aab343fb55 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:05.879984 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf38899-9c06-40ec-a969-31aab343fb55 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:05.891857 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea87666-2d54-4f81-b972-0ffbd773db6e None admin] RBAC: Authorizing `identity:create_grant(project_id=d5398feee6eb490cb49bb261448a1a9b, user_id=6409b68b4832421ba590d7bc53131f7c, role_id=f5641817ee56438496afc6135d37a4c0)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:05.893601 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea87666-2d54-4f81-b972-0ffbd773db6e None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:05.899506 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 301/597] 10.222.0.23 () {56 vars in 1139 bytes} [Fri May 6 14:45:05 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:05.900304 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf38899-9c06-40ec-a969-31aab343fb55 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:05.908652 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 297/598] 10.222.0.23 () {58 vars in 1439 bytes} [Fri May 6 14:45:05 2022] PUT /identity/v3/projects/d5398feee6eb490cb49bb261448a1a9b/users/6409b68b4832421ba590d7bc53131f7c/roles/f5641817ee56438496afc6135d37a4c0 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:45:05.909759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea87666-2d54-4f81-b972-0ffbd773db6e None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:06.013930 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf38899-9c06-40ec-a969-31aab343fb55 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:06.055463 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1240ce04-f4dc-46e9-9ed3-3e315f4af467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:06.056298 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240ce04-f4dc-46e9-9ed3-3e315f4af467 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:06.057537 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240ce04-f4dc-46e9-9ed3-3e315f4af467 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:06.058427 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240ce04-f4dc-46e9-9ed3-3e315f4af467 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:06.059288 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1240ce04-f4dc-46e9-9ed3-3e315f4af467 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:06.059876 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1240ce04-f4dc-46e9-9ed3-3e315f4af467 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:06.061931 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea87666-2d54-4f81-b972-0ffbd773db6e None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:06.064549 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1240ce04-f4dc-46e9-9ed3-3e315f4af467 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:06.090768 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 302/599] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:06.097899 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1240ce04-f4dc-46e9-9ed3-3e315f4af467 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:06.144630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3387b878-dd01-4681-90d5-9efae9a6cb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:06.145437 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387b878-dd01-4681-90d5-9efae9a6cb17 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:06.145714 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387b878-dd01-4681-90d5-9efae9a6cb17 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:06.145990 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387b878-dd01-4681-90d5-9efae9a6cb17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:06.160564 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3387b878-dd01-4681-90d5-9efae9a6cb17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:06.162125 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3387b878-dd01-4681-90d5-9efae9a6cb17 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:06.189108 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-3387b878-dd01-4681-90d5-9efae9a6cb17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:06.189108 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:06.189108 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:06.189108 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:06.189108 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:06.189108 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:06.189108 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:06.189108 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:06.189108 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:06.189108 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:06.189108 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:06.189108 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:06.189108 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:06.189108 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:06.189108 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:06.190266 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:06.190266 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:06.190266 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:06.190266 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:06.190266 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:06.190266 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:06.190266 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:06.190266 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:06.190266 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:06.190266 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:06.190266 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:06.190266 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:06.190266 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:06.190266 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:06.190266 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:06.190266 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:06.190266 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:06.191371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:06.191371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:06.191371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:06.191371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:06.194334 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 298/600] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:06.200393 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3387b878-dd01-4681-90d5-9efae9a6cb17 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:06.253266 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1240ce04-f4dc-46e9-9ed3-3e315f4af467 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:06.325259 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9dabf0-7665-48db-bef3-71f1fe0555c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:06.326116 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9dabf0-7665-48db-bef3-71f1fe0555c1 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:06.326404 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9dabf0-7665-48db-bef3-71f1fe0555c1 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:06.326680 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9dabf0-7665-48db-bef3-71f1fe0555c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:06.341999 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9dabf0-7665-48db-bef3-71f1fe0555c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:06.343444 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9dabf0-7665-48db-bef3-71f1fe0555c1 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:06.370986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-ba9dabf0-7665-48db-bef3-71f1fe0555c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:06.370986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:06.370986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:06.370986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:06.370986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:06.370986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:06.370986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:06.370986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:06.370986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:06.370986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:06.370986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:06.370986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:06.370986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 06 14:45:06.370986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(project_id) May 06 14:45:06.370986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 06 14:45:06.371910 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 06 14:45:06.371910 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:06.371910 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:06.371910 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:06.371910 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:06.371910 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:06.371910 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:06.371910 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:06.371910 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:06.371910 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:06.371910 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:06.371910 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:06.371910 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:06.371910 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:06.371910 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:06.371910 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:06.371910 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:06.372857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 06 14:45:06.372857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 06 14:45:06.372857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 06 14:45:06.372857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 06 14:45:06.372857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 06 14:45:06.372857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 06 14:45:06.372857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:06.372857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:06.372857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 303/601] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:45:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:06.381581 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3387b878-dd01-4681-90d5-9efae9a6cb17 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:06.383506 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9dabf0-7665-48db-bef3-71f1fe0555c1 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:06.429068 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e09f1ef-de0d-4273-b606-9938520cdc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:06.429747 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e09f1ef-de0d-4273-b606-9938520cdc0f None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:06.430004 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e09f1ef-de0d-4273-b606-9938520cdc0f None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:06.430274 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e09f1ef-de0d-4273-b606-9938520cdc0f None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:06.430999 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e09f1ef-de0d-4273-b606-9938520cdc0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:06.431305 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e09f1ef-de0d-4273-b606-9938520cdc0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:06.432713 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e09f1ef-de0d-4273-b606-9938520cdc0f None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:06.445630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 299/602] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:06.446971 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1264e1-2251-4d57-be32-a7d17dc16580 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:06.447263 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1264e1-2251-4d57-be32-a7d17dc16580 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:06.447542 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1264e1-2251-4d57-be32-a7d17dc16580 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:06.449132 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 300/603] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:45:06.450387 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1264e1-2251-4d57-be32-a7d17dc16580 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:06.500570 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9dabf0-7665-48db-bef3-71f1fe0555c1 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:06.535903 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41e9f4-7adb-4cfe-a067-acd03c35185d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:06.536678 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41e9f4-7adb-4cfe-a067-acd03c35185d None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:06.536896 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41e9f4-7adb-4cfe-a067-acd03c35185d None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:06.540631 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41e9f4-7adb-4cfe-a067-acd03c35185d None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:06.540631 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d41e9f4-7adb-4cfe-a067-acd03c35185d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:06.540631 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d41e9f4-7adb-4cfe-a067-acd03c35185d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:06.540631 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d41e9f4-7adb-4cfe-a067-acd03c35185d None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:06.552531 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 304/604] 10.222.0.23 () {56 vars in 1148 bytes} [Fri May 6 14:45:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:06.554822 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5987b2-e91f-4a72-94cb-3969dddece6d None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:06.555096 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5987b2-e91f-4a72-94cb-3969dddece6d None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:06.555375 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5987b2-e91f-4a72-94cb-3969dddece6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:06.555674 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1264e1-2251-4d57-be32-a7d17dc16580 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:06.573659 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-5a5987b2-e91f-4a72-94cb-3969dddece6d None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:06.574834 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-5a5987b2-e91f-4a72-94cb-3969dddece6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:06.597280 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4f83b9-64f6-4738-b7ad-c2d7e2e3ade6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:06.598004 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f83b9-64f6-4738-b7ad-c2d7e2e3ade6 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:06.598287 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f83b9-64f6-4738-b7ad-c2d7e2e3ade6 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:06.598562 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f83b9-64f6-4738-b7ad-c2d7e2e3ade6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:06.606525 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4f83b9-64f6-4738-b7ad-c2d7e2e3ade6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:06.607856 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4f83b9-64f6-4738-b7ad-c2d7e2e3ade6 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:06.622483 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-7c4f83b9-64f6-4738-b7ad-c2d7e2e3ade6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:06.622483 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:06.622483 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:06.622483 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:06.622483 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:06.622483 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:06.622483 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:06.622483 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:06.622483 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:06.622483 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:06.622483 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:06.622483 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:06.622483 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 06 14:45:06.622483 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(project_id) May 06 14:45:06.622483 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 06 14:45:06.623401 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 06 14:45:06.623401 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:06.623401 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:06.623401 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:06.623401 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:06.623401 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:06.623401 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:06.623401 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:06.623401 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:06.623401 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:06.623401 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:06.623401 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:06.623401 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:06.623401 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:06.623401 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:06.623401 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:06.623401 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:06.624269 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 06 14:45:06.624269 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 06 14:45:06.624269 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 06 14:45:06.624269 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 06 14:45:06.624269 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 06 14:45:06.624269 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 06 14:45:06.624269 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:06.624269 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:06.624269 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 301/605] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:45:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:06.625245 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4f83b9-64f6-4738-b7ad-c2d7e2e3ade6 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:06.656705 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 305/606] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:45:06.657587 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5987b2-e91f-4a72-94cb-3969dddece6d None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:06.746377 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4f83b9-64f6-4738-b7ad-c2d7e2e3ade6 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:06.781005 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b68fea-fa12-4830-849e-f17030023118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:06.781926 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b68fea-fa12-4830-849e-f17030023118 None admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:06.782208 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b68fea-fa12-4830-849e-f17030023118 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:06.782482 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b68fea-fa12-4830-849e-f17030023118 None admin] PATH_INFO: `/v3/projects/d5398feee6eb490cb49bb261448a1a9b/users/d9be93f9ff7e46babc8ee87b50b9c5eb/roles/f5641817ee56438496afc6135d37a4c0` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:06.792353 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5987b2-e91f-4a72-94cb-3969dddece6d None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:06.797794 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b68fea-fa12-4830-849e-f17030023118 None admin] RBAC: Authorizing `identity:create_grant(project_id=d5398feee6eb490cb49bb261448a1a9b, user_id=d9be93f9ff7e46babc8ee87b50b9c5eb, role_id=f5641817ee56438496afc6135d37a4c0)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:06.799377 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b68fea-fa12-4830-849e-f17030023118 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:06.817519 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 302/607] 10.222.0.23 () {58 vars in 1439 bytes} [Fri May 6 14:45:06 2022] PUT /identity/v3/projects/d5398feee6eb490cb49bb261448a1a9b/users/d9be93f9ff7e46babc8ee87b50b9c5eb/roles/f5641817ee56438496afc6135d37a4c0 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:45:06.819181 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b68fea-fa12-4830-849e-f17030023118 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:06.831383 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22396250-30e0-4be8-8fbf-807c2e27396a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:06.832221 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22396250-30e0-4be8-8fbf-807c2e27396a None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:06.832570 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22396250-30e0-4be8-8fbf-807c2e27396a None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:06.832974 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22396250-30e0-4be8-8fbf-807c2e27396a None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:06.834443 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22396250-30e0-4be8-8fbf-807c2e27396a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:06.834823 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22396250-30e0-4be8-8fbf-807c2e27396a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:06.836581 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22396250-30e0-4be8-8fbf-807c2e27396a None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:06.851704 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 306/608] 10.222.0.23 () {56 vars in 1148 bytes} [Fri May 6 14:45:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:06.857693 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22396250-30e0-4be8-8fbf-807c2e27396a None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:06.949244 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b68fea-fa12-4830-849e-f17030023118 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:06.989817 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22396250-30e0-4be8-8fbf-807c2e27396a None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:07.030849 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57bc0fa-4643-4a8c-8919-308a326f9265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:07.032140 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57bc0fa-4643-4a8c-8919-308a326f9265 None admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:07.032429 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57bc0fa-4643-4a8c-8919-308a326f9265 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:07.032744 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57bc0fa-4643-4a8c-8919-308a326f9265 None admin] PATH_INFO: `/v3/projects/d5398feee6eb490cb49bb261448a1a9b/users/4dcd1542d8f643c4996142a88b9b6abb/roles/f5641817ee56438496afc6135d37a4c0` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:07.047687 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b885962-67cd-4a54-a288-4783d14d1d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:07.048569 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b885962-67cd-4a54-a288-4783d14d1d04 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:07.049003 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b885962-67cd-4a54-a288-4783d14d1d04 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:07.049438 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b885962-67cd-4a54-a288-4783d14d1d04 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:07.050317 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57bc0fa-4643-4a8c-8919-308a326f9265 None admin] RBAC: Authorizing `identity:create_grant(project_id=d5398feee6eb490cb49bb261448a1a9b, user_id=4dcd1542d8f643c4996142a88b9b6abb, role_id=f5641817ee56438496afc6135d37a4c0)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:07.052155 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57bc0fa-4643-4a8c-8919-308a326f9265 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:07.069582 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 307/609] 10.222.0.23 () {58 vars in 1439 bytes} [Fri May 6 14:45:06 2022] PUT /identity/v3/projects/d5398feee6eb490cb49bb261448a1a9b/users/4dcd1542d8f643c4996142a88b9b6abb/roles/f5641817ee56438496afc6135d37a4c0 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 14:45:07.073676 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b885962-67cd-4a54-a288-4783d14d1d04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:45:07.081701 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-9b885962-67cd-4a54-a288-4783d14d1d04 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 06 14:45:07.081701 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:07.081701 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:07.081701 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:07.081701 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:07.081701 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:07.081701 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:07.081701 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:07.081701 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:07.081701 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:07.081701 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:07.081701 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:07.081701 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:45:07.081701 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:45:07.081701 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:45:07.081701 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:45:07.082704 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:45:07.082704 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:45:07.082704 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:45:07.082704 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:45:07.082704 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:07.082704 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:07.082704 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:07.082704 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:07.082704 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:07.082704 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:07.082704 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:07.082704 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:07.082704 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:07.082704 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:07.082704 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:07.082704 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:07.083571 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:07.083571 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:07.083571 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:07.083571 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:07.083571 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:45:07.083571 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:45:07.083571 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:45:07.083571 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:45:07.083571 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:45:07.083571 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:45:07.083571 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 06 14:45:07.083571 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:07.083571 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 303/610] 10.222.0.23 () {56 vars in 1102 bytes} [Fri May 6 14:45:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:07.089715 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57bc0fa-4643-4a8c-8919-308a326f9265 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:07.221475 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57bc0fa-4643-4a8c-8919-308a326f9265 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:07.270412 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8005afa-aa59-40dd-aa47-299d9c2e79c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:07.271989 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8005afa-aa59-40dd-aa47-299d9c2e79c2 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:07.272432 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8005afa-aa59-40dd-aa47-299d9c2e79c2 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:07.272877 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8005afa-aa59-40dd-aa47-299d9c2e79c2 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:07.273814 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8005afa-aa59-40dd-aa47-299d9c2e79c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:07.274275 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8005afa-aa59-40dd-aa47-299d9c2e79c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:07.275816 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8005afa-aa59-40dd-aa47-299d9c2e79c2 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:07.289763 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 308/611] 10.222.0.23 () {56 vars in 1103 bytes} [Fri May 6 14:45:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:07.295222 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b885962-67cd-4a54-a288-4783d14d1d04 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:07.412111 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b885962-67cd-4a54-a288-4783d14d1d04 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:07.436424 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aefe0d3-6b8a-4261-8fdc-38482e9b49d5 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:07.436755 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aefe0d3-6b8a-4261-8fdc-38482e9b49d5 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:07.437018 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aefe0d3-6b8a-4261-8fdc-38482e9b49d5 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:07.439163 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 309/612] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:45:07.445304 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7899ccf5-97fc-4434-a8e5-26fa1cf09a24 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:07.445599 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7899ccf5-97fc-4434-a8e5-26fa1cf09a24 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:07.445885 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7899ccf5-97fc-4434-a8e5-26fa1cf09a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:07.465934 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3558b3e5-1a65-415f-817e-40bc9fc47d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:07.466735 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3558b3e5-1a65-415f-817e-40bc9fc47d3f None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:07.467018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3558b3e5-1a65-415f-817e-40bc9fc47d3f None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:07.467291 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3558b3e5-1a65-415f-817e-40bc9fc47d3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:07.469629 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-7899ccf5-97fc-4434-a8e5-26fa1cf09a24 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:07.470823 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-7899ccf5-97fc-4434-a8e5-26fa1cf09a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:07.482898 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3558b3e5-1a65-415f-817e-40bc9fc47d3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:07.484356 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3558b3e5-1a65-415f-817e-40bc9fc47d3f None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:07.502222 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-3558b3e5-1a65-415f-817e-40bc9fc47d3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:07.502222 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:07.502222 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:07.502222 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:07.502222 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:07.502222 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:07.502222 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:07.502222 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:07.502222 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:07.502222 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:07.502222 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:07.502222 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:07.502222 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:07.502222 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:07.502222 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:07.503159 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:07.503159 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:07.503159 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:07.503159 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:07.503159 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:07.503159 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:07.503159 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:07.503159 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:07.503159 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:07.503159 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:07.503159 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:07.503159 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:07.503159 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:07.503159 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:07.503159 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:07.503159 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:07.503159 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:07.504462 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:07.504462 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:07.504462 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:07.504462 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:07.504462 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 304/613] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:07.511308 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3558b3e5-1a65-415f-817e-40bc9fc47d3f None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:07.566968 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 310/614] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 06 14:45:07.625627 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7899ccf5-97fc-4434-a8e5-26fa1cf09a24 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:07.686869 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3558b3e5-1a65-415f-817e-40bc9fc47d3f None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:07.748992 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7899ccf5-97fc-4434-a8e5-26fa1cf09a24 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:07.749981 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9aa2d-e3ce-4e73-8b84-6af58514d729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:07.750873 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9aa2d-e3ce-4e73-8b84-6af58514d729 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:07.751367 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9aa2d-e3ce-4e73-8b84-6af58514d729 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:07.751850 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9aa2d-e3ce-4e73-8b84-6af58514d729 None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:07.752796 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e9aa2d-e3ce-4e73-8b84-6af58514d729 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:07.753316 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e9aa2d-e3ce-4e73-8b84-6af58514d729 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:07.754899 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e9aa2d-e3ce-4e73-8b84-6af58514d729 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:07.781027 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 305/615] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:07.782666 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9aa2d-e3ce-4e73-8b84-6af58514d729 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:07.793318 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cbc33b-a1f3-43d3-8529-35e792b50639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:07.794132 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbc33b-a1f3-43d3-8529-35e792b50639 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:07.794675 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbc33b-a1f3-43d3-8529-35e792b50639 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:07.794675 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbc33b-a1f3-43d3-8529-35e792b50639 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:07.825570 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cbc33b-a1f3-43d3-8529-35e792b50639 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:45:07.841362 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-61cbc33b-a1f3-43d3-8529-35e792b50639 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 06 14:45:07.841362 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:07.841362 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:07.841362 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:07.841362 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:07.841362 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:07.841362 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:07.841362 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:07.841362 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:07.841362 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:07.841362 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:07.841362 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:07.841362 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:45:07.841362 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:45:07.841362 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:45:07.841362 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:45:07.842352 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:45:07.842352 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:45:07.842352 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:45:07.842352 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:45:07.842352 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:07.842352 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:07.842352 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:07.842352 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:07.842352 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:07.842352 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:07.842352 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:07.842352 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:07.842352 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:07.842352 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:07.842352 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:07.842352 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:07.843832 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:07.843832 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:07.843832 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:07.843832 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:07.843832 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:45:07.843832 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:45:07.843832 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:45:07.843832 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:45:07.843832 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:45:07.843832 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:45:07.843832 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 06 14:45:07.843832 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:07.843832 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 311/616] 10.222.0.23 () {56 vars in 1102 bytes} [Fri May 6 14:45:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:45:07.848392 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cbc33b-a1f3-43d3-8529-35e792b50639 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:07.889965 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9aa2d-e3ce-4e73-8b84-6af58514d729 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:07.937415 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c403b0-50b0-4574-8403-8edc219419d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:07.938582 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c403b0-50b0-4574-8403-8edc219419d7 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:07.939067 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c403b0-50b0-4574-8403-8edc219419d7 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:07.939542 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c403b0-50b0-4574-8403-8edc219419d7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:07.947650 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c403b0-50b0-4574-8403-8edc219419d7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:07.956622 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c403b0-50b0-4574-8403-8edc219419d7 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:07.962278 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cbc33b-a1f3-43d3-8529-35e792b50639 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:07.979636 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-39c403b0-50b0-4574-8403-8edc219419d7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 06 14:45:07.979636 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:07.979636 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:07.979636 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:07.979636 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:07.979636 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:07.979636 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:07.979636 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:07.979636 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:07.979636 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:07.979636 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:07.979636 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:07.979636 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 06 14:45:07.979636 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_user(user_id) May 06 14:45:07.979636 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 06 14:45:07.979636 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 06 14:45:07.980659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:07.980659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:07.980659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 06 14:45:07.980659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:07.980659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 06 14:45:07.980659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:07.980659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:07.980659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:07.980659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:07.980659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:07.980659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:07.980659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:07.980659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:07.980659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:07.980659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:07.980659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:07.980659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:07.980659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:07.981626 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:07.981626 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:07.981626 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 06 14:45:07.981626 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 06 14:45:07.981626 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 06 14:45:07.981626 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 06 14:45:07.981626 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 06 14:45:07.981626 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 06 14:45:07.981626 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 06 14:45:07.981626 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:07.993028 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 306/617] 10.222.0.23 () {56 vars in 1096 bytes} [Fri May 6 14:45:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:07.994185 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c403b0-50b0-4574-8403-8edc219419d7 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:08.009356 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c4544-6a5e-4ac8-b8f9-007b053eebb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:08.010106 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c4544-6a5e-4ac8-b8f9-007b053eebb1 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:08.010400 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c4544-6a5e-4ac8-b8f9-007b053eebb1 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:08.010684 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c4544-6a5e-4ac8-b8f9-007b053eebb1 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:08.011441 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437c4544-6a5e-4ac8-b8f9-007b053eebb1 None admin] RBAC: Adding query filter params (name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:08.011753 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437c4544-6a5e-4ac8-b8f9-007b053eebb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:08.013217 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437c4544-6a5e-4ac8-b8f9-007b053eebb1 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:08.027940 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 312/618] 10.222.0.23 () {56 vars in 1103 bytes} [Fri May 6 14:45:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:08.033301 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c4544-6a5e-4ac8-b8f9-007b053eebb1 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:08.098114 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c403b0-50b0-4574-8403-8edc219419d7 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:08.142657 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef555cd-1c63-406a-a2e5-4963d3367f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:08.143707 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef555cd-1c63-406a-a2e5-4963d3367f39 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:08.144190 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef555cd-1c63-406a-a2e5-4963d3367f39 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:08.144686 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef555cd-1c63-406a-a2e5-4963d3367f39 None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:08.145723 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef555cd-1c63-406a-a2e5-4963d3367f39 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:08.146373 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef555cd-1c63-406a-a2e5-4963d3367f39 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:08.148698 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef555cd-1c63-406a-a2e5-4963d3367f39 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:08.156757 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c4544-6a5e-4ac8-b8f9-007b053eebb1 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:08.187407 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 307/619] 10.222.0.23 () {56 vars in 1135 bytes} [Fri May 6 14:45:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:08.194289 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef555cd-1c63-406a-a2e5-4963d3367f39 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:08.199461 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee17f35-e4ac-44e1-8fbd-bb8a30e51fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:08.200584 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee17f35-e4ac-44e1-8fbd-bb8a30e51fe3 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:08.203014 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee17f35-e4ac-44e1-8fbd-bb8a30e51fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:08.203542 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee17f35-e4ac-44e1-8fbd-bb8a30e51fe3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:08.227722 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee17f35-e4ac-44e1-8fbd-bb8a30e51fe3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:08.229718 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee17f35-e4ac-44e1-8fbd-bb8a30e51fe3 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:08.260138 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-0ee17f35-e4ac-44e1-8fbd-bb8a30e51fe3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:08.260138 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:08.260138 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:08.260138 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:08.260138 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:08.260138 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:08.260138 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:08.260138 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:08.260138 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:08.260138 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:08.260138 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:08.260138 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:08.260138 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:08.260138 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:08.260138 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:08.261157 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:08.261157 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:08.261157 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:08.261157 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:08.261157 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:08.261157 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:08.261157 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:08.261157 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:08.261157 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:08.261157 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:08.261157 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:08.261157 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:08.261157 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:08.261157 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:08.261157 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:08.261157 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:08.261157 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:08.262053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:08.262053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:08.262053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:08.262053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:08.264450 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 313/620] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:45:08.272560 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee17f35-e4ac-44e1-8fbd-bb8a30e51fe3 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:08.386143 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef555cd-1c63-406a-a2e5-4963d3367f39 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:08.429889 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510995d-48b8-4222-b4ef-70607cfe8d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:08.430699 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510995d-48b8-4222-b4ef-70607cfe8d71 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:08.430982 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510995d-48b8-4222-b4ef-70607cfe8d71 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:08.431256 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510995d-48b8-4222-b4ef-70607cfe8d71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:08.450617 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7510995d-48b8-4222-b4ef-70607cfe8d71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:08.452010 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7510995d-48b8-4222-b4ef-70607cfe8d71 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:08.470084 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee17f35-e4ac-44e1-8fbd-bb8a30e51fe3 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:08.487230 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-7510995d-48b8-4222-b4ef-70607cfe8d71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:08.487230 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:08.487230 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:08.487230 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:08.487230 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:08.487230 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:08.487230 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:08.487230 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:08.487230 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:08.487230 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:08.487230 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:08.487230 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:08.487230 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:08.487230 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:08.487230 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:08.488510 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:08.488510 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:08.488510 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:08.488510 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:08.488510 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:08.488510 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:08.488510 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:08.488510 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:08.488510 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:08.488510 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:08.488510 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:08.488510 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:08.488510 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:08.488510 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:08.488510 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:08.488510 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:08.488510 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:08.489799 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:08.489799 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:08.489799 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:08.489799 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:08.489799 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 308/621] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:08.496795 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510995d-48b8-4222-b4ef-70607cfe8d71 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:08.519510 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78c5047-16ed-4476-8c55-b8280489232f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:08.520190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c5047-16ed-4476-8c55-b8280489232f None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:08.520473 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c5047-16ed-4476-8c55-b8280489232f None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:08.520774 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c5047-16ed-4476-8c55-b8280489232f None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:08.521542 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78c5047-16ed-4476-8c55-b8280489232f None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:08.521853 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78c5047-16ed-4476-8c55-b8280489232f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:08.523245 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78c5047-16ed-4476-8c55-b8280489232f None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:08.543384 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 314/622] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:08.544810 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0b009-2a13-4025-9682-a82d3c1eaa77 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:08.545106 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0b009-2a13-4025-9682-a82d3c1eaa77 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:08.545385 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0b009-2a13-4025-9682-a82d3c1eaa77 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:08.547023 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 315/623] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:45:08.548258 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af0b009-2a13-4025-9682-a82d3c1eaa77 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:08.641185 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510995d-48b8-4222-b4ef-70607cfe8d71 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:08.667606 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af0b009-2a13-4025-9682-a82d3c1eaa77 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:08.676989 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1883c943-fb1f-4286-9c53-89db74765e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:08.678642 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883c943-fb1f-4286-9c53-89db74765e39 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:08.678642 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883c943-fb1f-4286-9c53-89db74765e39 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:08.679080 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883c943-fb1f-4286-9c53-89db74765e39 None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:08.679901 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1883c943-fb1f-4286-9c53-89db74765e39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:08.680304 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1883c943-fb1f-4286-9c53-89db74765e39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:08.681801 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1883c943-fb1f-4286-9c53-89db74765e39 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:08.698168 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 309/624] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:08.699719 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c14f52-5a55-4f5e-af88-d3ca7057d5c0 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:08.700108 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c14f52-5a55-4f5e-af88-d3ca7057d5c0 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:08.700489 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c14f52-5a55-4f5e-af88-d3ca7057d5c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:08.715541 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0910503e-6c11-452e-8f75-dacb921d0e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:08.716797 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0910503e-6c11-452e-8f75-dacb921d0e3c None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:08.717235 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0910503e-6c11-452e-8f75-dacb921d0e3c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:08.717659 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0910503e-6c11-452e-8f75-dacb921d0e3c None admin] PATH_INFO: `/v3/users/nova` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:08.719528 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-63c14f52-5a55-4f5e-af88-d3ca7057d5c0 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:08.721271 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-63c14f52-5a55-4f5e-af88-d3ca7057d5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:08.729754 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0910503e-6c11-452e-8f75-dacb921d0e3c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:08.731934 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0910503e-6c11-452e-8f75-dacb921d0e3c None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:08.752120 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-0910503e-6c11-452e-8f75-dacb921d0e3c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 06 14:45:08.752120 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:08.752120 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:08.752120 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:08.752120 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:08.752120 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:08.752120 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:08.752120 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:08.752120 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:08.752120 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:08.752120 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:08.752120 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:08.752120 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 06 14:45:08.752120 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_user(user_id) May 06 14:45:08.752120 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 06 14:45:08.752120 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 06 14:45:08.753326 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:08.753326 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:08.753326 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 06 14:45:08.753326 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:08.753326 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 06 14:45:08.753326 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:08.753326 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:08.753326 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:08.753326 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:08.753326 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:08.753326 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:08.753326 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:08.753326 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:08.753326 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:08.753326 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:08.753326 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:08.753326 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:08.753326 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:08.754495 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:08.754495 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:08.754495 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 06 14:45:08.754495 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 06 14:45:08.754495 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 06 14:45:08.754495 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 06 14:45:08.754495 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 06 14:45:08.754495 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 06 14:45:08.754495 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 06 14:45:08.754495 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:08.757499 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 316/625] 10.222.0.23 () {56 vars in 1093 bytes} [Fri May 6 14:45:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:45:08.758444 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0910503e-6c11-452e-8f75-dacb921d0e3c None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:08.801581 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 310/626] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:45:08.805475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c14f52-5a55-4f5e-af88-d3ca7057d5c0 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:08.873631 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0910503e-6c11-452e-8f75-dacb921d0e3c None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:08.913403 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2746a09-e0f7-4ddc-9fc0-6affe105a0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:08.914372 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2746a09-e0f7-4ddc-9fc0-6affe105a0d7 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:08.914751 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2746a09-e0f7-4ddc-9fc0-6affe105a0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:08.915165 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2746a09-e0f7-4ddc-9fc0-6affe105a0d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:08.915996 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c14f52-5a55-4f5e-af88-d3ca7057d5c0 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:08.922124 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2746a09-e0f7-4ddc-9fc0-6affe105a0d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:08.923681 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2746a09-e0f7-4ddc-9fc0-6affe105a0d7 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:08.938191 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-f2746a09-e0f7-4ddc-9fc0-6affe105a0d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:08.938191 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:08.938191 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:08.938191 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:08.938191 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:08.938191 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:08.938191 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:08.938191 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:08.938191 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:08.938191 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:08.938191 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:08.938191 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:08.938191 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 06 14:45:08.938191 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(project_id) May 06 14:45:08.938191 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 06 14:45:08.939329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 06 14:45:08.939329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:08.939329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:08.939329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:08.939329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:08.939329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:08.939329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:08.939329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:08.939329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:08.939329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:08.939329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:08.939329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:08.939329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:08.939329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:08.939329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:08.939329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:08.939329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:08.940395 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 06 14:45:08.940395 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 06 14:45:08.940395 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 06 14:45:08.940395 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 06 14:45:08.940395 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 06 14:45:08.940395 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 06 14:45:08.940395 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:08.940395 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:08.947342 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 317/627] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:45:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:08.948340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2746a09-e0f7-4ddc-9fc0-6affe105a0d7 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:08.957201 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e31235-8bfa-446b-971f-bef334848863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:08.958073 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e31235-8bfa-446b-971f-bef334848863 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:08.959472 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e31235-8bfa-446b-971f-bef334848863 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:08.959903 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e31235-8bfa-446b-971f-bef334848863 None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:08.960935 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e31235-8bfa-446b-971f-bef334848863 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:08.961748 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e31235-8bfa-446b-971f-bef334848863 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:08.963837 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e31235-8bfa-446b-971f-bef334848863 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:08.987456 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 311/628] 10.222.0.23 () {56 vars in 1133 bytes} [Fri May 6 14:45:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:08.988524 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e31235-8bfa-446b-971f-bef334848863 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:09.051022 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2746a09-e0f7-4ddc-9fc0-6affe105a0d7 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:09.087035 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73d28a-ff1e-4ad5-bbb0-465517288225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:09.088084 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73d28a-ff1e-4ad5-bbb0-465517288225 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:09.088486 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73d28a-ff1e-4ad5-bbb0-465517288225 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:09.088893 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73d28a-ff1e-4ad5-bbb0-465517288225 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:09.100428 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f73d28a-ff1e-4ad5-bbb0-465517288225 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:45:09.108518 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-1f73d28a-ff1e-4ad5-bbb0-465517288225 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 06 14:45:09.108518 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:09.108518 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:09.108518 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:09.108518 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:09.108518 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:09.108518 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:09.108518 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:09.108518 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:09.108518 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:09.108518 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:09.108518 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:09.108518 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:45:09.108518 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:45:09.108518 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:45:09.108518 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:45:09.109677 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:45:09.109677 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:45:09.109677 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:45:09.109677 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:45:09.109677 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:09.109677 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:09.109677 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:09.109677 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:09.109677 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:09.109677 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:09.109677 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:09.109677 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:09.109677 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:09.109677 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:09.109677 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:09.109677 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:09.110684 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:09.110684 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:09.110684 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:09.110684 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:09.110684 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:45:09.110684 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:45:09.110684 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:45:09.110684 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:45:09.110684 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:45:09.110684 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:45:09.110684 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 06 14:45:09.110684 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:09.113030 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 318/629] 10.222.0.23 () {56 vars in 1102 bytes} [Fri May 6 14:45:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:09.114126 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73d28a-ff1e-4ad5-bbb0-465517288225 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:09.119588 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e31235-8bfa-446b-971f-bef334848863 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:09.162871 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f69de-26da-4c71-8d36-92486fa1c6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:09.163887 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f69de-26da-4c71-8d36-92486fa1c6e2 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:09.164328 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f69de-26da-4c71-8d36-92486fa1c6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:09.164806 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f69de-26da-4c71-8d36-92486fa1c6e2 None admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:09.165775 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f69de-26da-4c71-8d36-92486fa1c6e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:09.166235 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f69de-26da-4c71-8d36-92486fa1c6e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:09.168023 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f69de-26da-4c71-8d36-92486fa1c6e2 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:09.189203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 312/630] 10.222.0.23 () {56 vars in 1148 bytes} [Fri May 6 14:45:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:09.189203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f69de-26da-4c71-8d36-92486fa1c6e2 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:09.232385 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73d28a-ff1e-4ad5-bbb0-465517288225 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:09.275467 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b7e265-1c2e-4e46-bd78-e29f022247a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:09.276361 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7e265-1c2e-4e46-bd78-e29f022247a9 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:09.276835 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7e265-1c2e-4e46-bd78-e29f022247a9 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:09.277734 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7e265-1c2e-4e46-bd78-e29f022247a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:09.309022 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f69de-26da-4c71-8d36-92486fa1c6e2 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:09.343992 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4b80d-04ff-4a2d-ac95-6737de867075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:09.344818 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4b80d-04ff-4a2d-ac95-6737de867075 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:09.345217 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4b80d-04ff-4a2d-ac95-6737de867075 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:09.345599 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4b80d-04ff-4a2d-ac95-6737de867075 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:09.346453 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae4b80d-04ff-4a2d-ac95-6737de867075 None admin] RBAC: Adding query filter params (name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:09.346867 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae4b80d-04ff-4a2d-ac95-6737de867075 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:09.348552 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae4b80d-04ff-4a2d-ac95-6737de867075 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:09.357568 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 313/631] 10.222.0.23 () {56 vars in 1103 bytes} [Fri May 6 14:45:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:09.358605 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4b80d-04ff-4a2d-ac95-6737de867075 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:09.367788 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b7e265-1c2e-4e46-bd78-e29f022247a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:09.369988 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b7e265-1c2e-4e46-bd78-e29f022247a9 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:09.478910 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4b80d-04ff-4a2d-ac95-6737de867075 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:09.482178 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 319/632] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:45:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 06 14:45:09.483230 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b7e265-1c2e-4e46-bd78-e29f022247a9 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:09.527423 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d6c24-0696-4689-b3cd-1d48224f8fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:09.529042 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d6c24-0696-4689-b3cd-1d48224f8fbf None admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:09.529480 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d6c24-0696-4689-b3cd-1d48224f8fbf None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:09.530856 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d6c24-0696-4689-b3cd-1d48224f8fbf None admin] PATH_INFO: `/v3/projects/d5398feee6eb490cb49bb261448a1a9b/users/bf9eed62c8f1409282c2af6488426a0f/roles/f5641817ee56438496afc6135d37a4c0` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:09.554352 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110d6c24-0696-4689-b3cd-1d48224f8fbf None admin] RBAC: Authorizing `identity:create_grant(project_id=d5398feee6eb490cb49bb261448a1a9b, user_id=bf9eed62c8f1409282c2af6488426a0f, role_id=f5641817ee56438496afc6135d37a4c0)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:09.555821 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110d6c24-0696-4689-b3cd-1d48224f8fbf None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:09.580627 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 314/633] 10.222.0.23 () {58 vars in 1439 bytes} [Fri May 6 14:45:09 2022] PUT /identity/v3/projects/d5398feee6eb490cb49bb261448a1a9b/users/bf9eed62c8f1409282c2af6488426a0f/roles/f5641817ee56438496afc6135d37a4c0 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:45:09.580627 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d6c24-0696-4689-b3cd-1d48224f8fbf None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:09.639117 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b7e265-1c2e-4e46-bd78-e29f022247a9 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:09.707305 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da1fd0-df49-404a-bb4a-7a7ef7e4ab2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:09.708122 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da1fd0-df49-404a-bb4a-7a7ef7e4ab2d None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:09.708405 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da1fd0-df49-404a-bb4a-7a7ef7e4ab2d None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:09.708804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da1fd0-df49-404a-bb4a-7a7ef7e4ab2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:09.720744 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5da1fd0-df49-404a-bb4a-7a7ef7e4ab2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:09.721511 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5da1fd0-df49-404a-bb4a-7a7ef7e4ab2d None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:09.742338 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-f5da1fd0-df49-404a-bb4a-7a7ef7e4ab2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:09.742338 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:09.742338 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:09.742338 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:09.742338 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:09.742338 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:09.742338 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:09.742338 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:09.742338 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:09.742338 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:09.742338 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:09.742338 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:09.742338 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:09.742338 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:09.742338 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:09.743346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:09.743346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:09.743346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:09.743346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:09.743346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:09.743346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:09.743346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:09.743346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:09.743346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:09.743346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:09.743346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:09.743346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:09.743346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:09.743346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:09.743346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:09.743346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:09.743346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:09.744228 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:09.744228 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:09.744228 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:09.744228 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:09.746616 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 320/634] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:09.747831 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c1cb8-d3b4-4d79-8ea8-f9b3c6b00f04 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:09.748137 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c1cb8-d3b4-4d79-8ea8-f9b3c6b00f04 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:09.748626 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c1cb8-d3b4-4d79-8ea8-f9b3c6b00f04 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:09.749793 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 321/635] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:45:09.750924 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff0ce32-c152-42b1-a415-d0516764d3e7 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:09.752878 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff0ce32-c152-42b1-a415-d0516764d3e7 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:09.753705 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff0ce32-c152-42b1-a415-d0516764d3e7 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:09.755390 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 322/636] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:45:09.756555 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff0ce32-c152-42b1-a415-d0516764d3e7 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:09.765169 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d6c24-0696-4689-b3cd-1d48224f8fbf None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:09.808210 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff53fd65-1795-48ea-a828-11b6ad13d1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:09.808981 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53fd65-1795-48ea-a828-11b6ad13d1d4 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:09.809262 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53fd65-1795-48ea-a828-11b6ad13d1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:09.809539 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53fd65-1795-48ea-a828-11b6ad13d1d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:09.820340 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff53fd65-1795-48ea-a828-11b6ad13d1d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:09.821836 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff53fd65-1795-48ea-a828-11b6ad13d1d4 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:09.839745 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-ff53fd65-1795-48ea-a828-11b6ad13d1d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:09.839745 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:09.839745 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:09.839745 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:09.839745 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:09.839745 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:09.839745 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:09.839745 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:09.839745 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:09.839745 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:09.839745 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:09.839745 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:09.839745 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:09.839745 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:09.839745 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:09.840780 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:09.840780 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:09.840780 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:09.840780 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:09.840780 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:09.840780 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:09.840780 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:09.840780 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:09.840780 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:09.840780 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:09.840780 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:09.840780 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:09.840780 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:09.840780 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:09.840780 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:09.840780 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:09.840780 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:09.841681 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:09.841681 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:09.841681 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:09.841681 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:09.841681 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 315/637] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:09.842512 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d31891-aec8-48ab-92d8-d9615b473496 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:09.846221 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d31891-aec8-48ab-92d8-d9615b473496 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:09.846221 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d31891-aec8-48ab-92d8-d9615b473496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:09.861889 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-a5d31891-aec8-48ab-92d8-d9615b473496 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:09.863071 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-a5d31891-aec8-48ab-92d8-d9615b473496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:09.872357 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff0ce32-c152-42b1-a415-d0516764d3e7 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:09.907229 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aec243-1f09-480b-b7cf-1e05ff988f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:09.907902 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aec243-1f09-480b-b7cf-1e05ff988f46 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:09.908182 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aec243-1f09-480b-b7cf-1e05ff988f46 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:09.908458 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aec243-1f09-480b-b7cf-1e05ff988f46 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:09.909218 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aec243-1f09-480b-b7cf-1e05ff988f46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:09.909527 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aec243-1f09-480b-b7cf-1e05ff988f46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:09.910896 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aec243-1f09-480b-b7cf-1e05ff988f46 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:09.926925 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 323/638] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:09.926925 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e890d9-99a2-4eb2-b768-d0f68ea5525e None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:09.926925 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e890d9-99a2-4eb2-b768-d0f68ea5525e None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:09.926925 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e890d9-99a2-4eb2-b768-d0f68ea5525e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:09.935617 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 316/639] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:45:09.936303 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d31891-aec8-48ab-92d8-d9615b473496 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:09.948632 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-14e890d9-99a2-4eb2-b768-d0f68ea5525e None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:09.948632 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-14e890d9-99a2-4eb2-b768-d0f68ea5525e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:10.027958 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 324/640] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:45:10.029004 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e890d9-99a2-4eb2-b768-d0f68ea5525e None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:10.047343 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d31891-aec8-48ab-92d8-d9615b473496 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:10.083681 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4946fd-57e5-444e-a4a3-c3102bd51687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:10.084306 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4946fd-57e5-444e-a4a3-c3102bd51687 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:10.084680 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4946fd-57e5-444e-a4a3-c3102bd51687 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:10.084952 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4946fd-57e5-444e-a4a3-c3102bd51687 None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:10.085712 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4946fd-57e5-444e-a4a3-c3102bd51687 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:10.086022 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4946fd-57e5-444e-a4a3-c3102bd51687 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:10.087403 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4946fd-57e5-444e-a4a3-c3102bd51687 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:10.100256 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 317/641] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:10.101136 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4946fd-57e5-444e-a4a3-c3102bd51687 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:10.138334 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e890d9-99a2-4eb2-b768-d0f68ea5525e None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:10.178255 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dfd362-8a98-4590-b1f5-bbbffcbab33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:10.179065 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dfd362-8a98-4590-b1f5-bbbffcbab33f None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:10.179348 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dfd362-8a98-4590-b1f5-bbbffcbab33f None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:10.179626 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dfd362-8a98-4590-b1f5-bbbffcbab33f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:10.187034 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dfd362-8a98-4590-b1f5-bbbffcbab33f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:10.188476 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dfd362-8a98-4590-b1f5-bbbffcbab33f None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:10.203662 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-38dfd362-8a98-4590-b1f5-bbbffcbab33f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 06 14:45:10.203662 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:10.203662 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:10.203662 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:10.203662 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:10.203662 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:10.203662 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:10.203662 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:10.203662 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:10.203662 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:10.203662 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:10.203662 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:10.203662 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 06 14:45:10.203662 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_user(user_id) May 06 14:45:10.203662 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 06 14:45:10.203662 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 06 14:45:10.204894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:10.204894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:10.204894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 06 14:45:10.204894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:10.204894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 06 14:45:10.204894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:10.204894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:10.204894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:10.204894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:10.204894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:10.204894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:10.204894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:10.204894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:10.204894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:10.204894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:10.204894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:10.204894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:10.204894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:10.205845 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:10.205845 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:10.205845 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 06 14:45:10.205845 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 06 14:45:10.205845 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 06 14:45:10.205845 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 06 14:45:10.205845 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 06 14:45:10.205845 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 06 14:45:10.205845 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 06 14:45:10.205845 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:10.205845 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 325/642] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:45:10 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:10.205845 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dfd362-8a98-4590-b1f5-bbbffcbab33f None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:10.216311 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4946fd-57e5-444e-a4a3-c3102bd51687 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:10.250367 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f8ff45-52b1-4b39-a06f-80e4873a0010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:10.251214 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8ff45-52b1-4b39-a06f-80e4873a0010 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:10.251494 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8ff45-52b1-4b39-a06f-80e4873a0010 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:10.251824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8ff45-52b1-4b39-a06f-80e4873a0010 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:10.262556 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f8ff45-52b1-4b39-a06f-80e4873a0010 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:45:10.270113 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-d6f8ff45-52b1-4b39-a06f-80e4873a0010 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 06 14:45:10.270113 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:10.270113 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:10.270113 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:10.270113 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:10.270113 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:10.270113 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:10.270113 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:10.270113 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:10.270113 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:10.270113 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:10.270113 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:10.270113 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:45:10.270113 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:45:10.270113 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:45:10.270113 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:45:10.271319 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:45:10.271319 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:45:10.271319 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:45:10.271319 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:45:10.271319 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:10.271319 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:10.271319 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:10.271319 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:10.271319 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:10.271319 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:10.271319 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:10.271319 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:10.271319 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:10.271319 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:10.271319 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:10.271319 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:10.272199 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:10.272199 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:10.272199 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:10.272199 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:10.272199 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:45:10.272199 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:45:10.272199 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:45:10.272199 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:45:10.272199 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:45:10.272199 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:45:10.272199 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 06 14:45:10.272199 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:10.276780 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 318/643] 10.222.0.23 () {56 vars in 1102 bytes} [Fri May 6 14:45:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:10.277563 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f8ff45-52b1-4b39-a06f-80e4873a0010 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:10.308245 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dfd362-8a98-4590-b1f5-bbbffcbab33f None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:10.342520 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7406653-13dc-431a-ab7f-09e0e563ac46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:10.343232 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7406653-13dc-431a-ab7f-09e0e563ac46 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:10.343571 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7406653-13dc-431a-ab7f-09e0e563ac46 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:10.343845 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7406653-13dc-431a-ab7f-09e0e563ac46 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:10.355458 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7406653-13dc-431a-ab7f-09e0e563ac46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:45:10.363190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-f7406653-13dc-431a-ab7f-09e0e563ac46 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 06 14:45:10.363190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:10.363190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:10.363190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:10.363190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:10.363190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:10.363190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:10.363190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:10.363190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:10.363190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:10.363190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:10.363190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:10.363190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:45:10.363190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:45:10.363190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:45:10.363190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:45:10.364171 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:45:10.364171 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:45:10.364171 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:45:10.364171 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:45:10.364171 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:10.364171 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:10.364171 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:10.364171 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:10.364171 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:10.364171 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:10.364171 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:10.364171 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:10.364171 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:10.364171 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:10.364171 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:10.364171 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:10.365778 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:10.365778 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:10.365778 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:10.365778 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:10.365778 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:45:10.365778 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:45:10.365778 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:45:10.365778 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:45:10.365778 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:45:10.365778 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:45:10.365778 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 06 14:45:10.365778 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:10.365778 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 326/644] 10.222.0.23 () {56 vars in 1102 bytes} [Fri May 6 14:45:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:10.366628 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7406653-13dc-431a-ab7f-09e0e563ac46 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:10.383521 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f8ff45-52b1-4b39-a06f-80e4873a0010 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:10.417160 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e2119-23f3-4608-8f8a-fb2dde338c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:10.417950 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e2119-23f3-4608-8f8a-fb2dde338c79 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:10.418227 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e2119-23f3-4608-8f8a-fb2dde338c79 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:10.418495 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e2119-23f3-4608-8f8a-fb2dde338c79 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:10.425435 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e2119-23f3-4608-8f8a-fb2dde338c79 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:10.426851 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e2119-23f3-4608-8f8a-fb2dde338c79 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:10.442032 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-de3e2119-23f3-4608-8f8a-fb2dde338c79 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:10.442032 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:10.442032 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:10.442032 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:10.442032 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:10.442032 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:10.442032 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:10.442032 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:10.442032 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:10.442032 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:10.442032 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:10.442032 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:10.442032 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 06 14:45:10.442032 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(project_id) May 06 14:45:10.442032 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 06 14:45:10.442964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 06 14:45:10.442964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:10.442964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:10.442964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:10.442964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:10.442964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:10.442964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:10.442964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:10.442964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:10.442964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:10.442964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:10.442964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:10.442964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:10.442964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:10.442964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:10.442964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:10.442964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:10.443930 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 06 14:45:10.443930 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 06 14:45:10.443930 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 06 14:45:10.443930 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 06 14:45:10.443930 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 06 14:45:10.443930 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 06 14:45:10.443930 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:10.443930 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:10.448586 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 319/645] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:45:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:10.449484 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e2119-23f3-4608-8f8a-fb2dde338c79 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:10.475683 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7406653-13dc-431a-ab7f-09e0e563ac46 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:10.509280 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b91140-2525-4492-b2cc-4962a9dad771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:10.510078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b91140-2525-4492-b2cc-4962a9dad771 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:10.510508 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b91140-2525-4492-b2cc-4962a9dad771 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:10.510663 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b91140-2525-4492-b2cc-4962a9dad771 None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:10.511465 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b91140-2525-4492-b2cc-4962a9dad771 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:10.511776 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b91140-2525-4492-b2cc-4962a9dad771 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:10.513418 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b91140-2525-4492-b2cc-4962a9dad771 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:10.532658 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 327/646] 10.222.0.23 () {56 vars in 1137 bytes} [Fri May 6 14:45:10 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:10.533897 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b91140-2525-4492-b2cc-4962a9dad771 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:10.549269 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e2119-23f3-4608-8f8a-fb2dde338c79 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:10.582880 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b589551-9054-48a6-94d4-a15d75736b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:10.584221 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b589551-9054-48a6-94d4-a15d75736b33 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:10.584881 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b589551-9054-48a6-94d4-a15d75736b33 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:10.584996 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b589551-9054-48a6-94d4-a15d75736b33 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:10.585823 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b589551-9054-48a6-94d4-a15d75736b33 None admin] RBAC: Adding query filter params (name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:10.586232 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b589551-9054-48a6-94d4-a15d75736b33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:10.587674 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b589551-9054-48a6-94d4-a15d75736b33 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:10.601406 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 320/647] 10.222.0.23 () {56 vars in 1103 bytes} [Fri May 6 14:45:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:10.601406 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b589551-9054-48a6-94d4-a15d75736b33 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:10.632811 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b91140-2525-4492-b2cc-4962a9dad771 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:10.667362 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44dd7ce-9aa9-48a8-90ce-9e887cf1be7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:10.668025 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44dd7ce-9aa9-48a8-90ce-9e887cf1be7a None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:10.668312 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44dd7ce-9aa9-48a8-90ce-9e887cf1be7a None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:10.668583 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44dd7ce-9aa9-48a8-90ce-9e887cf1be7a None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:10.669336 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44dd7ce-9aa9-48a8-90ce-9e887cf1be7a None admin] RBAC: Adding query filter params (name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:10.669637 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44dd7ce-9aa9-48a8-90ce-9e887cf1be7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:10.670992 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44dd7ce-9aa9-48a8-90ce-9e887cf1be7a None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:10.678953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 328/648] 10.222.0.23 () {56 vars in 1103 bytes} [Fri May 6 14:45:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:10.679834 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44dd7ce-9aa9-48a8-90ce-9e887cf1be7a None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:10.705715 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b589551-9054-48a6-94d4-a15d75736b33 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:10.749460 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6690c4e-ad52-48f1-a044-f7b52c496d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:10.750272 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6690c4e-ad52-48f1-a044-f7b52c496d66 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:10.750647 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6690c4e-ad52-48f1-a044-f7b52c496d66 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:10.751038 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6690c4e-ad52-48f1-a044-f7b52c496d66 None admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:10.751900 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6690c4e-ad52-48f1-a044-f7b52c496d66 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:10.752304 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6690c4e-ad52-48f1-a044-f7b52c496d66 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:10.753780 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6690c4e-ad52-48f1-a044-f7b52c496d66 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:10.765941 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 321/649] 10.222.0.23 () {56 vars in 1148 bytes} [Fri May 6 14:45:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:10.766747 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6690c4e-ad52-48f1-a044-f7b52c496d66 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:10.802109 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44dd7ce-9aa9-48a8-90ce-9e887cf1be7a None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:10.838173 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebfe15b-e276-44e2-b3f5-727849df9e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:10.838970 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebfe15b-e276-44e2-b3f5-727849df9e04 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:10.839501 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebfe15b-e276-44e2-b3f5-727849df9e04 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:10.840649 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebfe15b-e276-44e2-b3f5-727849df9e04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:10.871277 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6690c4e-ad52-48f1-a044-f7b52c496d66 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:10.910738 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7bd470-40a3-4936-b7cb-be802a173080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:10.912410 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7bd470-40a3-4936-b7cb-be802a173080 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:10.912929 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7bd470-40a3-4936-b7cb-be802a173080 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:10.913416 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7bd470-40a3-4936-b7cb-be802a173080 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:10.915530 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebfe15b-e276-44e2-b3f5-727849df9e04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:10.917071 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebfe15b-e276-44e2-b3f5-727849df9e04 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:10.924117 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7bd470-40a3-4936-b7cb-be802a173080 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:10.925632 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7bd470-40a3-4936-b7cb-be802a173080 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:10.944031 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-2c7bd470-40a3-4936-b7cb-be802a173080 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:10.944031 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:10.944031 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:10.944031 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:10.944031 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:10.944031 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:10.944031 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:10.944031 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:10.944031 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:10.944031 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:10.944031 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:10.944031 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:10.944031 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:10.944031 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:10.944031 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:10.945163 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:10.945163 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:10.945163 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:10.945163 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:10.945163 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:10.945163 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:10.945163 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:10.945163 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:10.945163 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:10.945163 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:10.945163 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:10.945163 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:10.945163 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:10.945163 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:10.945163 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:10.945163 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:10.945163 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:10.946056 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:10.946056 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:10.946056 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:10.946056 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:10.946056 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 322/650] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:10.946056 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7bd470-40a3-4936-b7cb-be802a173080 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:11.025649 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 329/651] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:45:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 06 14:45:11.025649 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebfe15b-e276-44e2-b3f5-727849df9e04 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:11.050632 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7bd470-40a3-4936-b7cb-be802a173080 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:11.084788 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e58865-639b-45d1-8baf-fb5caf957961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:11.085886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e58865-639b-45d1-8baf-fb5caf957961 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:11.086173 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e58865-639b-45d1-8baf-fb5caf957961 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:11.086480 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e58865-639b-45d1-8baf-fb5caf957961 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:11.125357 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebfe15b-e276-44e2-b3f5-727849df9e04 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:11.165329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a75c10-ac37-471a-8fa8-eb287532edc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:11.166006 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a75c10-ac37-471a-8fa8-eb287532edc2 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:11.166287 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a75c10-ac37-471a-8fa8-eb287532edc2 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:11.166560 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a75c10-ac37-471a-8fa8-eb287532edc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:11.167404 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a75c10-ac37-471a-8fa8-eb287532edc2 None admin] RBAC: Adding query filter params (role.id=f5641817ee56438496afc6135d37a4c0, scope.project.id=d5398feee6eb490cb49bb261448a1a9b, user.id=fc8cf09642ec4e10a80badcf8b585ef8) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:11.167716 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a75c10-ac37-471a-8fa8-eb287532edc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:11.169242 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a75c10-ac37-471a-8fa8-eb287532edc2 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:11.180974 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 330/652] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:45:11 2022] GET /identity/v3/role_assignments?user.id=fc8cf09642ec4e10a80badcf8b585ef8&scope.project.id=d5398feee6eb490cb49bb261448a1a9b&role.id=f5641817ee56438496afc6135d37a4c0 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:11.180974 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a75c10-ac37-471a-8fa8-eb287532edc2 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:11.221739 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e58865-639b-45d1-8baf-fb5caf957961 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:11.223238 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e58865-639b-45d1-8baf-fb5caf957961 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:11.242203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-53e58865-639b-45d1-8baf-fb5caf957961 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:11.242203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:11.242203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:11.242203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:11.242203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:11.242203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:11.242203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:11.242203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:11.242203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:11.242203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:11.242203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:11.242203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:11.242203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:11.242203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:11.242203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:11.243176 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:11.243176 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:11.243176 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:11.243176 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:11.243176 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:11.243176 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:11.243176 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:11.243176 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:11.243176 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:11.243176 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:11.243176 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:11.243176 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:11.243176 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:11.243176 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:11.243176 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:11.243176 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:11.243176 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:11.244057 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:11.244057 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:11.244057 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:11.244057 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:11.248493 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 323/653] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:11.249551 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e58865-639b-45d1-8baf-fb5caf957961 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:11.301325 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a75c10-ac37-471a-8fa8-eb287532edc2 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:11.339564 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dad7f6-ccda-4fd3-9e25-37cc3d0e6343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:11.340252 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dad7f6-ccda-4fd3-9e25-37cc3d0e6343 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:11.340534 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dad7f6-ccda-4fd3-9e25-37cc3d0e6343 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:11.340841 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dad7f6-ccda-4fd3-9e25-37cc3d0e6343 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:11.341570 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dad7f6-ccda-4fd3-9e25-37cc3d0e6343 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:11.341873 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dad7f6-ccda-4fd3-9e25-37cc3d0e6343 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:11.343249 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dad7f6-ccda-4fd3-9e25-37cc3d0e6343 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:11.356951 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 331/654] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:11.357845 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dad7f6-ccda-4fd3-9e25-37cc3d0e6343 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:11.363607 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e58865-639b-45d1-8baf-fb5caf957961 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:11.401278 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f5d796-3ecd-4c58-9b6a-0d4ad9ed84ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:11.402008 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f5d796-3ecd-4c58-9b6a-0d4ad9ed84ca None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:11.402290 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f5d796-3ecd-4c58-9b6a-0d4ad9ed84ca None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:11.402619 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f5d796-3ecd-4c58-9b6a-0d4ad9ed84ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:11.413642 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f5d796-3ecd-4c58-9b6a-0d4ad9ed84ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:11.415107 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f5d796-3ecd-4c58-9b6a-0d4ad9ed84ca None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:11.433872 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-c4f5d796-3ecd-4c58-9b6a-0d4ad9ed84ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:11.433872 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:11.433872 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:11.433872 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:11.433872 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:11.433872 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:11.433872 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:11.433872 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:11.433872 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:11.433872 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:11.433872 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:11.433872 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:11.433872 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:11.433872 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:11.433872 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:11.435953 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:11.435953 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:11.435953 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:11.435953 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:11.435953 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:11.435953 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:11.435953 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:11.435953 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:11.435953 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:11.435953 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:11.435953 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:11.435953 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:11.435953 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:11.435953 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:11.435953 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:11.435953 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:11.435953 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:11.438382 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:11.438382 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:11.438382 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:11.438382 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:11.438382 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 324/655] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:11.438382 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f5d796-3ecd-4c58-9b6a-0d4ad9ed84ca None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:11.466217 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dad7f6-ccda-4fd3-9e25-37cc3d0e6343 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:11.506385 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8214fea-ab06-47ee-bbe3-c327905d1354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:11.507324 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8214fea-ab06-47ee-bbe3-c327905d1354 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:11.507948 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8214fea-ab06-47ee-bbe3-c327905d1354 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:11.508235 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8214fea-ab06-47ee-bbe3-c327905d1354 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:11.509275 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8214fea-ab06-47ee-bbe3-c327905d1354 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:11.509753 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8214fea-ab06-47ee-bbe3-c327905d1354 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:11.511937 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8214fea-ab06-47ee-bbe3-c327905d1354 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:11.527744 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 332/656] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:11.532994 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8214fea-ab06-47ee-bbe3-c327905d1354 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:11.561083 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f5d796-3ecd-4c58-9b6a-0d4ad9ed84ca None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:11.596334 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a8b2c-4359-4496-90d4-26cd464ea9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:11.597171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a8b2c-4359-4496-90d4-26cd464ea9e0 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:11.597455 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a8b2c-4359-4496-90d4-26cd464ea9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:11.597736 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a8b2c-4359-4496-90d4-26cd464ea9e0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:11.605663 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76a8b2c-4359-4496-90d4-26cd464ea9e0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:11.607114 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76a8b2c-4359-4496-90d4-26cd464ea9e0 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:11.622622 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-a76a8b2c-4359-4496-90d4-26cd464ea9e0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 06 14:45:11.622622 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:11.622622 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:11.622622 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:11.622622 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:11.622622 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:11.622622 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:11.622622 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:11.622622 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:11.622622 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:11.622622 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:11.622622 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:11.622622 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 06 14:45:11.622622 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_user(user_id) May 06 14:45:11.622622 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 06 14:45:11.622622 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 06 14:45:11.623606 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:11.623606 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:11.623606 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 06 14:45:11.623606 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:11.623606 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 06 14:45:11.623606 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:11.623606 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:11.623606 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:11.623606 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:11.623606 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:11.623606 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:11.623606 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:11.623606 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:11.623606 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:11.623606 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:11.623606 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:11.623606 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:11.623606 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:11.624533 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:11.624533 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:11.624533 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 06 14:45:11.624533 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 06 14:45:11.624533 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 06 14:45:11.624533 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 06 14:45:11.624533 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 06 14:45:11.624533 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 06 14:45:11.624533 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 06 14:45:11.624533 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:11.624533 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 325/657] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:45:11 2022] GET /identity/v3/users/glance => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:11.629431 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a8b2c-4359-4496-90d4-26cd464ea9e0 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:11.637897 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8214fea-ab06-47ee-bbe3-c327905d1354 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:11.673762 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd211ddd-5265-4428-a704-c133c34447ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:11.674352 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd211ddd-5265-4428-a704-c133c34447ca None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:11.674633 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd211ddd-5265-4428-a704-c133c34447ca None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:11.674909 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd211ddd-5265-4428-a704-c133c34447ca None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:11.675629 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd211ddd-5265-4428-a704-c133c34447ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:11.675932 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd211ddd-5265-4428-a704-c133c34447ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:11.677336 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd211ddd-5265-4428-a704-c133c34447ca None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:11.690704 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 333/658] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:11.691583 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd211ddd-5265-4428-a704-c133c34447ca None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:11.748531 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a8b2c-4359-4496-90d4-26cd464ea9e0 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:11.784706 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd6aec6-cb42-4393-9a06-3299b782056c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:11.785517 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6aec6-cb42-4393-9a06-3299b782056c None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:11.785795 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6aec6-cb42-4393-9a06-3299b782056c None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:11.786069 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6aec6-cb42-4393-9a06-3299b782056c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:11.796003 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd6aec6-cb42-4393-9a06-3299b782056c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:11.797464 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd6aec6-cb42-4393-9a06-3299b782056c None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:11.802121 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd211ddd-5265-4428-a704-c133c34447ca None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:11.812203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-3cd6aec6-cb42-4393-9a06-3299b782056c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 06 14:45:11.812203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:11.812203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:11.812203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:11.812203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:11.812203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:11.812203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:11.812203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:11.812203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:11.812203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:11.812203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:11.812203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:11.812203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 06 14:45:11.812203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_user(user_id) May 06 14:45:11.812203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 06 14:45:11.812203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 06 14:45:11.814469 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:11.814469 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:11.814469 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 06 14:45:11.814469 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:11.814469 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 06 14:45:11.814469 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:11.814469 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:11.814469 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:11.814469 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:11.814469 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:11.814469 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:11.814469 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:11.814469 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:11.814469 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:11.814469 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:11.814469 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:11.814469 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:11.814469 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:11.816886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:11.816886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:11.816886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 06 14:45:11.816886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 06 14:45:11.816886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 06 14:45:11.816886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 06 14:45:11.816886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 06 14:45:11.816886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 06 14:45:11.816886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 06 14:45:11.816886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:11.816886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 326/659] 10.222.0.23 () {56 vars in 1102 bytes} [Fri May 6 14:45:11 2022] GET /identity/v3/users/neutron => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:11.816886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd6aec6-cb42-4393-9a06-3299b782056c None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:11.850300 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49980a5-a441-470f-8a70-7ec4b8eb304d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:11.851830 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49980a5-a441-470f-8a70-7ec4b8eb304d None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:11.852113 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49980a5-a441-470f-8a70-7ec4b8eb304d None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:11.852390 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49980a5-a441-470f-8a70-7ec4b8eb304d None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:11.853306 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49980a5-a441-470f-8a70-7ec4b8eb304d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:11.853617 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49980a5-a441-470f-8a70-7ec4b8eb304d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:11.855076 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49980a5-a441-470f-8a70-7ec4b8eb304d None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:11.880029 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 334/660] 10.222.0.23 () {56 vars in 1137 bytes} [Fri May 6 14:45:11 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:11.885194 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49980a5-a441-470f-8a70-7ec4b8eb304d None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:11.921837 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd6aec6-cb42-4393-9a06-3299b782056c None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:11.956274 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e5cd34-6992-42b3-90ee-165d82f2ae7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:11.956988 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5cd34-6992-42b3-90ee-165d82f2ae7e None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:11.957266 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5cd34-6992-42b3-90ee-165d82f2ae7e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:11.957533 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5cd34-6992-42b3-90ee-165d82f2ae7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:11.965424 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e5cd34-6992-42b3-90ee-165d82f2ae7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:11.966862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e5cd34-6992-42b3-90ee-165d82f2ae7e None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:11.981787 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-b3e5cd34-6992-42b3-90ee-165d82f2ae7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:11.981787 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:11.981787 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:11.981787 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:11.981787 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:11.981787 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:11.981787 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:11.981787 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:11.981787 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:11.981787 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:11.981787 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:11.981787 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:11.981787 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 06 14:45:11.981787 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(project_id) May 06 14:45:11.981787 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 06 14:45:11.982757 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 06 14:45:11.982757 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:11.982757 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:11.982757 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:11.982757 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:11.982757 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:11.982757 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:11.982757 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:11.982757 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:11.982757 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:11.982757 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:11.982757 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:11.982757 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:11.982757 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:11.982757 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:11.982757 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:11.982757 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:11.983696 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 06 14:45:11.983696 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 06 14:45:11.983696 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 06 14:45:11.983696 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 06 14:45:11.983696 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 06 14:45:11.983696 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 06 14:45:11.983696 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:11.983696 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:11.983696 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 327/661] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:45:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:11.984992 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e5cd34-6992-42b3-90ee-165d82f2ae7e None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:11.998304 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49980a5-a441-470f-8a70-7ec4b8eb304d None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:12.032522 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc262ce-bf72-4b08-a58d-b82a41221e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:12.033571 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc262ce-bf72-4b08-a58d-b82a41221e88 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:12.034020 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc262ce-bf72-4b08-a58d-b82a41221e88 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:12.034432 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc262ce-bf72-4b08-a58d-b82a41221e88 None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:12.035436 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc262ce-bf72-4b08-a58d-b82a41221e88 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:12.035894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc262ce-bf72-4b08-a58d-b82a41221e88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:12.038485 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc262ce-bf72-4b08-a58d-b82a41221e88 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:12.061755 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 335/662] 10.222.0.23 () {56 vars in 1139 bytes} [Fri May 6 14:45:11 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:12.061755 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c6537-134c-4e15-be2f-21350424523f None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:12.061755 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c6537-134c-4e15-be2f-21350424523f None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:12.061755 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c6537-134c-4e15-be2f-21350424523f None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:12.061755 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 336/663] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:45:12.061755 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c6537-134c-4e15-be2f-21350424523f None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:12.094856 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e5cd34-6992-42b3-90ee-165d82f2ae7e None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:12.129473 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c31872-3137-4328-b794-41fa52a958c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:12.130216 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c31872-3137-4328-b794-41fa52a958c6 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:12.130596 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c31872-3137-4328-b794-41fa52a958c6 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:12.131085 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c31872-3137-4328-b794-41fa52a958c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:12.167605 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c6537-134c-4e15-be2f-21350424523f None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:12.198781 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c31872-3137-4328-b794-41fa52a958c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:12.200301 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c31872-3137-4328-b794-41fa52a958c6 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:12.201617 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2174be30-65f0-4681-82fc-8c61ed1202f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:12.202393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174be30-65f0-4681-82fc-8c61ed1202f6 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:12.202782 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174be30-65f0-4681-82fc-8c61ed1202f6 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:12.203150 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174be30-65f0-4681-82fc-8c61ed1202f6 None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:12.204025 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2174be30-65f0-4681-82fc-8c61ed1202f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:12.204431 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2174be30-65f0-4681-82fc-8c61ed1202f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:12.205973 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2174be30-65f0-4681-82fc-8c61ed1202f6 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:12.222960 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 337/664] 10.222.0.23 () {56 vars in 1148 bytes} [Fri May 6 14:45:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:12.222960 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2174be30-65f0-4681-82fc-8c61ed1202f6 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:12.302002 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 328/665] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:45:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 06 14:45:12.302002 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50135d9-e2da-4e2a-ae9e-2dde96ccf607 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:12.302002 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50135d9-e2da-4e2a-ae9e-2dde96ccf607 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:12.302002 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50135d9-e2da-4e2a-ae9e-2dde96ccf607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:12.317450 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-f50135d9-e2da-4e2a-ae9e-2dde96ccf607 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:12.321031 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-f50135d9-e2da-4e2a-ae9e-2dde96ccf607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:12.321443 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2174be30-65f0-4681-82fc-8c61ed1202f6 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:12.356017 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a7ddc-dc5b-40c2-b844-2bf9db115f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:12.356702 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a7ddc-dc5b-40c2-b844-2bf9db115f68 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:12.356993 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a7ddc-dc5b-40c2-b844-2bf9db115f68 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:12.357271 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a7ddc-dc5b-40c2-b844-2bf9db115f68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:12.394191 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 329/666] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:45:12.394191 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50135d9-e2da-4e2a-ae9e-2dde96ccf607 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:12.426701 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530a7ddc-dc5b-40c2-b844-2bf9db115f68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:12.428284 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530a7ddc-dc5b-40c2-b844-2bf9db115f68 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:12.487755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50135d9-e2da-4e2a-ae9e-2dde96ccf607 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:12.522546 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f2570-8b23-4359-b5cd-439d21a03a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:12.523217 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f2570-8b23-4359-b5cd-439d21a03a89 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:12.523500 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f2570-8b23-4359-b5cd-439d21a03a89 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:12.523772 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f2570-8b23-4359-b5cd-439d21a03a89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:12.524630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669f2570-8b23-4359-b5cd-439d21a03a89 None admin] RBAC: Adding query filter params (role.id=f5641817ee56438496afc6135d37a4c0, scope.project.id=d5398feee6eb490cb49bb261448a1a9b, user.id=6409b68b4832421ba590d7bc53131f7c) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:12.524938 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669f2570-8b23-4359-b5cd-439d21a03a89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:12.526358 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669f2570-8b23-4359-b5cd-439d21a03a89 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:12.531465 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 338/667] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:45:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 06 14:45:12.531465 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a7ddc-dc5b-40c2-b844-2bf9db115f68 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:12.536371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 330/668] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:45:12 2022] GET /identity/v3/role_assignments?user.id=6409b68b4832421ba590d7bc53131f7c&scope.project.id=d5398feee6eb490cb49bb261448a1a9b&role.id=f5641817ee56438496afc6135d37a4c0 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:12.537561 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f2570-8b23-4359-b5cd-439d21a03a89 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:12.651802 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f2570-8b23-4359-b5cd-439d21a03a89 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:12.655964 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a7ddc-dc5b-40c2-b844-2bf9db115f68 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:12.691129 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26630e75-a60b-4122-9fe9-d925e4e57bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:12.692587 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26630e75-a60b-4122-9fe9-d925e4e57bb6 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:12.693848 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26630e75-a60b-4122-9fe9-d925e4e57bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:12.694426 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26630e75-a60b-4122-9fe9-d925e4e57bb6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:12.694917 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c315fcd4-15dc-49fd-a1ab-828ff6e78671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:12.695886 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315fcd4-15dc-49fd-a1ab-828ff6e78671 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:12.696254 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315fcd4-15dc-49fd-a1ab-828ff6e78671 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:12.696618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315fcd4-15dc-49fd-a1ab-828ff6e78671 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:12.707560 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26630e75-a60b-4122-9fe9-d925e4e57bb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:45:12.707965 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c315fcd4-15dc-49fd-a1ab-828ff6e78671 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:12.710125 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c315fcd4-15dc-49fd-a1ab-828ff6e78671 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:12.717320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-26630e75-a60b-4122-9fe9-d925e4e57bb6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 06 14:45:12.717320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:12.717320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:12.717320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:12.717320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:12.717320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:12.717320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:12.717320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:12.717320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:12.717320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:12.717320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:12.717320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:12.717320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:45:12.717320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:45:12.717320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:45:12.717320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:45:12.718323 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:45:12.718323 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:45:12.718323 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:45:12.718323 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:45:12.718323 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:12.718323 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:12.718323 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:12.718323 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:12.718323 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:12.718323 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:12.718323 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:12.718323 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:12.718323 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:12.718323 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:12.718323 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:12.718323 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:12.719245 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:12.719245 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:12.719245 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:12.719245 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:12.719245 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:45:12.719245 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:45:12.719245 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:45:12.719245 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:45:12.719245 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:45:12.719245 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:45:12.719245 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 06 14:45:12.719245 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:12.722083 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 331/669] 10.222.0.23 () {56 vars in 1102 bytes} [Fri May 6 14:45:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:12.723083 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26630e75-a60b-4122-9fe9-d925e4e57bb6 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:12.730006 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-c315fcd4-15dc-49fd-a1ab-828ff6e78671 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:12.730006 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:12.730006 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:12.730006 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:12.730006 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:12.730006 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:12.730006 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:12.730006 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:12.730006 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:12.730006 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:12.730006 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:12.730006 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:12.730006 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:12.730006 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:12.730006 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:12.730894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:12.730894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:12.730894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:12.730894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:12.730894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:12.730894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:12.730894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:12.730894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:12.730894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:12.730894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:12.730894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:12.730894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:12.730894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:12.730894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:12.730894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:12.730894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:12.730894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:12.731899 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:12.731899 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:12.731899 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:12.731899 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:12.731899 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 339/670] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:12.733676 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c315fcd4-15dc-49fd-a1ab-828ff6e78671 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:12.845408 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c315fcd4-15dc-49fd-a1ab-828ff6e78671 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:12.848628 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26630e75-a60b-4122-9fe9-d925e4e57bb6 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:12.880720 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ad5866-fdd2-4349-a78b-8cee2f610a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:12.881995 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad5866-fdd2-4349-a78b-8cee2f610a51 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:12.882434 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad5866-fdd2-4349-a78b-8cee2f610a51 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:12.882859 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad5866-fdd2-4349-a78b-8cee2f610a51 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:12.882973 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead6e38-0669-429e-8c23-a28aa689032d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:12.883838 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead6e38-0669-429e-8c23-a28aa689032d None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:12.884030 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ad5866-fdd2-4349-a78b-8cee2f610a51 None admin] RBAC: Adding query filter params (name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:12.884254 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead6e38-0669-429e-8c23-a28aa689032d None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:12.884613 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead6e38-0669-429e-8c23-a28aa689032d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:12.884784 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ad5866-fdd2-4349-a78b-8cee2f610a51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:12.886292 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ad5866-fdd2-4349-a78b-8cee2f610a51 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:12.895303 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 340/671] 10.222.0.23 () {56 vars in 1103 bytes} [Fri May 6 14:45:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:12.896710 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bead6e38-0669-429e-8c23-a28aa689032d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:12.897114 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ad5866-fdd2-4349-a78b-8cee2f610a51 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:12.898522 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bead6e38-0669-429e-8c23-a28aa689032d None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:12.917774 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-bead6e38-0669-429e-8c23-a28aa689032d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:12.917774 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:12.917774 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:12.917774 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:12.917774 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:12.917774 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:12.917774 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:12.917774 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:12.917774 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:12.917774 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:12.917774 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:12.917774 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:12.917774 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:12.917774 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:12.917774 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:12.918956 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:12.918956 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:12.918956 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:12.918956 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:12.918956 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:12.918956 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:12.918956 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:12.918956 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:12.918956 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:12.918956 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:12.918956 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:12.918956 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:12.918956 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:12.918956 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:12.918956 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:12.918956 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:12.918956 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:12.920072 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:12.920072 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:12.920072 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:12.920072 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:12.921304 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 332/672] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:12.926471 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead6e38-0669-429e-8c23-a28aa689032d None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:13.018025 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ad5866-fdd2-4349-a78b-8cee2f610a51 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:13.030469 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead6e38-0669-429e-8c23-a28aa689032d None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:13.054706 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc905f-a9df-4879-afc4-8bd766977b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:13.055646 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc905f-a9df-4879-afc4-8bd766977b50 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:13.056070 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc905f-a9df-4879-afc4-8bd766977b50 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:13.056493 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc905f-a9df-4879-afc4-8bd766977b50 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:13.057397 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc905f-a9df-4879-afc4-8bd766977b50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:13.057854 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc905f-a9df-4879-afc4-8bd766977b50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:13.059357 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc905f-a9df-4879-afc4-8bd766977b50 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:13.066380 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c3f94a-f614-4277-9493-96211184f192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:13.067280 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c3f94a-f614-4277-9493-96211184f192 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:13.067629 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c3f94a-f614-4277-9493-96211184f192 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:13.068077 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c3f94a-f614-4277-9493-96211184f192 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:13.073164 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 341/673] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:13.074214 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc905f-a9df-4879-afc4-8bd766977b50 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:13.080253 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c3f94a-f614-4277-9493-96211184f192 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:13.081865 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c3f94a-f614-4277-9493-96211184f192 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:13.101303 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-06c3f94a-f614-4277-9493-96211184f192 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:13.101303 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:13.101303 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:13.101303 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:13.101303 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:13.101303 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:13.101303 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:13.101303 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:13.101303 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:13.101303 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:13.101303 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:13.101303 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:13.101303 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:13.101303 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:13.101303 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:13.102417 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:13.102417 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:13.102417 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:13.102417 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:13.102417 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:13.102417 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:13.102417 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:13.102417 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:13.102417 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:13.102417 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:13.102417 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:13.102417 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:13.102417 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:13.102417 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:13.102417 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:13.102417 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:13.102417 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:13.103540 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:13.103540 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:13.103540 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:13.103540 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:13.104632 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 333/674] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:13.105603 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c3f94a-f614-4277-9493-96211184f192 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:13.183808 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc905f-a9df-4879-afc4-8bd766977b50 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:13.223162 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a93d12-cf95-4c17-87f3-a993bc3d319a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:13.224028 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a93d12-cf95-4c17-87f3-a993bc3d319a None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:13.225140 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a93d12-cf95-4c17-87f3-a993bc3d319a None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:13.225596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a93d12-cf95-4c17-87f3-a993bc3d319a None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:13.226470 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a93d12-cf95-4c17-87f3-a993bc3d319a None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:13.226890 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a93d12-cf95-4c17-87f3-a993bc3d319a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:13.228552 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a93d12-cf95-4c17-87f3-a993bc3d319a None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:13.229548 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c3f94a-f614-4277-9493-96211184f192 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:13.242077 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 342/675] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:13.243183 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a93d12-cf95-4c17-87f3-a993bc3d319a None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:13.278599 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dae4d2-1c18-4832-a32b-e9396d88d02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:13.280018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dae4d2-1c18-4832-a32b-e9396d88d02c None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:13.280312 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dae4d2-1c18-4832-a32b-e9396d88d02c None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:13.280887 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dae4d2-1c18-4832-a32b-e9396d88d02c None admin] PATH_INFO: `/v3/projects/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:13.290148 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dae4d2-1c18-4832-a32b-e9396d88d02c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:13.291641 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dae4d2-1c18-4832-a32b-e9396d88d02c None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:13.306809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-a3dae4d2-1c18-4832-a32b-e9396d88d02c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:13.306809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:13.306809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:13.306809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:13.306809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:13.306809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:13.306809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:13.306809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:13.306809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:13.306809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:13.306809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:13.306809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:13.306809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 06 14:45:13.306809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(project_id) May 06 14:45:13.306809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 06 14:45:13.309022 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 06 14:45:13.309022 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:13.309022 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:13.309022 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:13.309022 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:13.309022 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:13.309022 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:13.309022 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:13.309022 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:13.309022 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:13.309022 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:13.309022 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:13.309022 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:13.309022 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:13.309022 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:13.309022 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:13.309022 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:13.311168 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 06 14:45:13.311168 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 06 14:45:13.311168 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 06 14:45:13.311168 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 06 14:45:13.311168 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 06 14:45:13.311168 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 06 14:45:13.311168 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:13.311168 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:13.311168 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 334/676] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:45:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:13.311168 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dae4d2-1c18-4832-a32b-e9396d88d02c None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:13.358269 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a93d12-cf95-4c17-87f3-a993bc3d319a None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:13.394140 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ed3c4-d967-40db-9231-abbb5ed4ee73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:13.394831 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ed3c4-d967-40db-9231-abbb5ed4ee73 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:13.395117 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ed3c4-d967-40db-9231-abbb5ed4ee73 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:13.395398 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ed3c4-d967-40db-9231-abbb5ed4ee73 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:13.396783 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6ed3c4-d967-40db-9231-abbb5ed4ee73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:13.397102 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6ed3c4-d967-40db-9231-abbb5ed4ee73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:13.398698 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6ed3c4-d967-40db-9231-abbb5ed4ee73 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:13.412550 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 343/677] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:13.413883 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ed3c4-d967-40db-9231-abbb5ed4ee73 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:13.415088 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dae4d2-1c18-4832-a32b-e9396d88d02c None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:13.452022 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e3bb17-6ff5-44da-9c7a-1f0dd751b862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:13.452909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e3bb17-6ff5-44da-9c7a-1f0dd751b862 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:13.453189 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e3bb17-6ff5-44da-9c7a-1f0dd751b862 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:13.453467 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e3bb17-6ff5-44da-9c7a-1f0dd751b862 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:13.461469 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e3bb17-6ff5-44da-9c7a-1f0dd751b862 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:13.462911 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e3bb17-6ff5-44da-9c7a-1f0dd751b862 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:13.478040 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-56e3bb17-6ff5-44da-9c7a-1f0dd751b862 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:13.478040 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:13.478040 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:13.478040 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:13.478040 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:13.478040 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:13.478040 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:13.478040 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:13.478040 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:13.478040 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:13.478040 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:13.478040 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:13.478040 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 06 14:45:13.478040 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(project_id) May 06 14:45:13.478040 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 06 14:45:13.480153 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 06 14:45:13.480153 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:13.480153 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:13.480153 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:13.480153 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:13.480153 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:13.480153 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:13.480153 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:13.480153 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:13.480153 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:13.480153 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:13.480153 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:13.480153 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:13.480153 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:13.480153 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:13.480153 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:13.480153 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:13.482312 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 06 14:45:13.482312 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 06 14:45:13.482312 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 06 14:45:13.482312 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 06 14:45:13.482312 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 06 14:45:13.482312 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 06 14:45:13.482312 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:13.482312 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:13.482312 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 335/678] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:45:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:13.482312 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e3bb17-6ff5-44da-9c7a-1f0dd751b862 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:13.542697 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ed3c4-d967-40db-9231-abbb5ed4ee73 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:13.585388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e3bb17-6ff5-44da-9c7a-1f0dd751b862 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:13.586256 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2c041-a3cb-4ed0-93de-dd13f1c8ca6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:13.586930 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2c041-a3cb-4ed0-93de-dd13f1c8ca6a None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:13.587391 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2c041-a3cb-4ed0-93de-dd13f1c8ca6a None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:13.587673 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2c041-a3cb-4ed0-93de-dd13f1c8ca6a None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:13.589105 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a2c041-a3cb-4ed0-93de-dd13f1c8ca6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:13.589424 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a2c041-a3cb-4ed0-93de-dd13f1c8ca6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:13.590912 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a2c041-a3cb-4ed0-93de-dd13f1c8ca6a None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:13.606325 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 344/679] 10.222.0.23 () {56 vars in 1148 bytes} [Fri May 6 14:45:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:13.607339 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2c041-a3cb-4ed0-93de-dd13f1c8ca6a None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:13.625182 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1515cd42-5687-4c15-a100-e2c6f01b5c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:13.626073 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515cd42-5687-4c15-a100-e2c6f01b5c95 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:13.626360 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515cd42-5687-4c15-a100-e2c6f01b5c95 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:13.627028 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515cd42-5687-4c15-a100-e2c6f01b5c95 None admin] PATH_INFO: `/v3/users/swift` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:13.636221 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1515cd42-5687-4c15-a100-e2c6f01b5c95 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:13.638142 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1515cd42-5687-4c15-a100-e2c6f01b5c95 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:13.652958 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-1515cd42-5687-4c15-a100-e2c6f01b5c95 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 06 14:45:13.652958 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:13.652958 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:13.652958 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:13.652958 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:13.652958 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:13.652958 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:13.652958 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:13.652958 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:13.652958 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:13.652958 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:13.652958 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:13.652958 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 06 14:45:13.652958 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_user(user_id) May 06 14:45:13.652958 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 06 14:45:13.652958 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 06 14:45:13.654391 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:13.654391 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:13.654391 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 06 14:45:13.654391 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:13.654391 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 06 14:45:13.654391 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:13.654391 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:13.654391 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:13.654391 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:13.654391 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:13.654391 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:13.654391 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:13.654391 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:13.654391 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:13.654391 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:13.654391 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:13.654391 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:13.654391 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:13.656018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:13.656018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:13.656018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 06 14:45:13.656018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 06 14:45:13.656018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 06 14:45:13.656018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 06 14:45:13.656018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 06 14:45:13.656018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 06 14:45:13.656018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 06 14:45:13.656018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:13.656018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 336/680] 10.222.0.23 () {56 vars in 1096 bytes} [Fri May 6 14:45:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:13.656018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc282864-bdf9-40bf-87f7-d4516bb8f475 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:13.656018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc282864-bdf9-40bf-87f7-d4516bb8f475 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:13.656772 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc282864-bdf9-40bf-87f7-d4516bb8f475 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:13.658752 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 337/681] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:45:13.659904 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc282864-bdf9-40bf-87f7-d4516bb8f475 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:13.714814 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2c041-a3cb-4ed0-93de-dd13f1c8ca6a None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:13.747489 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3eaf96-7679-454a-8b5a-6054230fa66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:13.748155 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3eaf96-7679-454a-8b5a-6054230fa66a None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:13.748442 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3eaf96-7679-454a-8b5a-6054230fa66a None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:13.748752 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3eaf96-7679-454a-8b5a-6054230fa66a None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:13.749534 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3eaf96-7679-454a-8b5a-6054230fa66a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:13.749843 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3eaf96-7679-454a-8b5a-6054230fa66a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:13.751268 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3eaf96-7679-454a-8b5a-6054230fa66a None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:13.763698 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 345/682] 10.222.0.23 () {56 vars in 1148 bytes} [Fri May 6 14:45:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:13.765076 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3eaf96-7679-454a-8b5a-6054230fa66a None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:13.766052 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc282864-bdf9-40bf-87f7-d4516bb8f475 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:13.807726 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c0406-ec7c-49e6-822d-a09375cad0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:13.808767 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c0406-ec7c-49e6-822d-a09375cad0be None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:13.809060 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c0406-ec7c-49e6-822d-a09375cad0be None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:13.809659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c0406-ec7c-49e6-822d-a09375cad0be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:13.810792 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7c0406-ec7c-49e6-822d-a09375cad0be None admin] RBAC: Adding query filter params (role.id=f5641817ee56438496afc6135d37a4c0, scope.project.id=d5398feee6eb490cb49bb261448a1a9b, user.id=d9be93f9ff7e46babc8ee87b50b9c5eb) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:13.811110 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7c0406-ec7c-49e6-822d-a09375cad0be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:13.813127 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7c0406-ec7c-49e6-822d-a09375cad0be None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:13.833030 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 338/683] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:45:13 2022] GET /identity/v3/role_assignments?user.id=d9be93f9ff7e46babc8ee87b50b9c5eb&scope.project.id=d5398feee6eb490cb49bb261448a1a9b&role.id=f5641817ee56438496afc6135d37a4c0 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:13.833030 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf454e0-bda6-4d72-b99d-16a213b6e426 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:13.833030 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf454e0-bda6-4d72-b99d-16a213b6e426 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:13.833030 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf454e0-bda6-4d72-b99d-16a213b6e426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:13.854369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-3bf454e0-bda6-4d72-b99d-16a213b6e426 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:13.855969 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-3bf454e0-bda6-4d72-b99d-16a213b6e426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:13.875053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3eaf96-7679-454a-8b5a-6054230fa66a None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:13.921866 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b405c-8995-41a0-af72-dd94bd01cd94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:13.922676 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b405c-8995-41a0-af72-dd94bd01cd94 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:13.923578 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b405c-8995-41a0-af72-dd94bd01cd94 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:13.923864 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b405c-8995-41a0-af72-dd94bd01cd94 None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:13.924767 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451b405c-8995-41a0-af72-dd94bd01cd94 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:13.925083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451b405c-8995-41a0-af72-dd94bd01cd94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:13.927068 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451b405c-8995-41a0-af72-dd94bd01cd94 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:13.944818 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 339/684] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:45:13.945892 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf454e0-bda6-4d72-b99d-16a213b6e426 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:13.951201 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 346/685] 10.222.0.23 () {56 vars in 1135 bytes} [Fri May 6 14:45:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:13.961368 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b405c-8995-41a0-af72-dd94bd01cd94 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:14.076574 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf454e0-bda6-4d72-b99d-16a213b6e426 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:14.087924 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b405c-8995-41a0-af72-dd94bd01cd94 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:14.122205 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c45485-b1b4-4b4f-9031-3b1ac897b7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:14.123022 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c45485-b1b4-4b4f-9031-3b1ac897b7f0 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:14.123402 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c45485-b1b4-4b4f-9031-3b1ac897b7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:14.123761 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c45485-b1b4-4b4f-9031-3b1ac897b7f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:14.125496 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c45485-b1b4-4b4f-9031-3b1ac897b7f0 None admin] RBAC: Adding query filter params (role.id=f5641817ee56438496afc6135d37a4c0, scope.project.id=d5398feee6eb490cb49bb261448a1a9b, user.id=4dcd1542d8f643c4996142a88b9b6abb) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:14.125496 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c45485-b1b4-4b4f-9031-3b1ac897b7f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:14.127231 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c45485-b1b4-4b4f-9031-3b1ac897b7f0 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:14.130963 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65393cd9-82f5-466b-a5c7-98ae16f751e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:14.131699 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65393cd9-82f5-466b-a5c7-98ae16f751e5 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:14.132059 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65393cd9-82f5-466b-a5c7-98ae16f751e5 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:14.132407 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65393cd9-82f5-466b-a5c7-98ae16f751e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:14.140701 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 340/686] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:45:13 2022] GET /identity/v3/role_assignments?user.id=4dcd1542d8f643c4996142a88b9b6abb&scope.project.id=d5398feee6eb490cb49bb261448a1a9b&role.id=f5641817ee56438496afc6135d37a4c0 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:14.141793 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c45485-b1b4-4b4f-9031-3b1ac897b7f0 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:14.215300 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65393cd9-82f5-466b-a5c7-98ae16f751e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:14.217270 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65393cd9-82f5-466b-a5c7-98ae16f751e5 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:14.282401 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c45485-b1b4-4b4f-9031-3b1ac897b7f0 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:14.318171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e177a-54fb-45a0-8fb2-fea6230eeee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:14.319501 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e177a-54fb-45a0-8fb2-fea6230eeee1 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:14.319997 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e177a-54fb-45a0-8fb2-fea6230eeee1 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:14.320478 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e177a-54fb-45a0-8fb2-fea6230eeee1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:14.331580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62e177a-54fb-45a0-8fb2-fea6230eeee1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:45:14.334690 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 347/687] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:45:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 06 14:45:14.340260 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65393cd9-82f5-466b-a5c7-98ae16f751e5 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:14.341440 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-d62e177a-54fb-45a0-8fb2-fea6230eeee1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:45:14.341440 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:14.341440 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:14.341440 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:14.341440 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:14.341440 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:14.341440 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:14.341440 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:14.341440 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:14.341440 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:14.341440 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:14.341440 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:14.341440 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:45:14.341440 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:45:14.341440 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:45:14.341440 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:45:14.342384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:45:14.342384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:45:14.342384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:45:14.342384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:45:14.342384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:14.342384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:14.342384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:14.342384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:14.342384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:14.342384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:14.342384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:14.342384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:14.342384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:14.342384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:14.342384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:14.342384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:14.343252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:14.343252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:14.343252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:14.343252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:14.343252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:45:14.343252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:45:14.343252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:45:14.343252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:45:14.343252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:45:14.343252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:45:14.343252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:45:14.343252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:14.345974 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 341/688] 10.222.0.23 () {56 vars in 1096 bytes} [Fri May 6 14:45:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:14.352719 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e177a-54fb-45a0-8fb2-fea6230eeee1 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:14.459739 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65393cd9-82f5-466b-a5c7-98ae16f751e5 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:14.491078 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e177a-54fb-45a0-8fb2-fea6230eeee1 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:14.501810 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826e5fa5-6ad0-4fb8-9acd-4b7467f59c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:14.502140 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e5fa5-6ad0-4fb8-9acd-4b7467f59c04 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:14.502358 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e5fa5-6ad0-4fb8-9acd-4b7467f59c04 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:14.502636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e5fa5-6ad0-4fb8-9acd-4b7467f59c04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:14.516125 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826e5fa5-6ad0-4fb8-9acd-4b7467f59c04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:14.517610 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826e5fa5-6ad0-4fb8-9acd-4b7467f59c04 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:14.534029 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a71773-af8c-4590-b8fd-0b65e23ef1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:14.535040 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a71773-af8c-4590-b8fd-0b65e23ef1e8 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:14.535546 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a71773-af8c-4590-b8fd-0b65e23ef1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:14.536013 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a71773-af8c-4590-b8fd-0b65e23ef1e8 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:14.537884 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a71773-af8c-4590-b8fd-0b65e23ef1e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:14.538500 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a71773-af8c-4590-b8fd-0b65e23ef1e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:14.540092 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a71773-af8c-4590-b8fd-0b65e23ef1e8 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:14.540731 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-826e5fa5-6ad0-4fb8-9acd-4b7467f59c04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:14.540731 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:14.540731 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:14.540731 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:14.540731 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:14.540731 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:14.540731 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:14.540731 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:14.540731 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:14.540731 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:14.540731 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:14.540731 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:14.540731 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:14.540731 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:14.540731 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:14.541651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:14.541651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:14.541651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:14.541651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:14.541651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:14.541651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:14.541651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:14.541651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:14.541651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:14.541651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:14.541651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:14.541651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:14.541651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:14.541651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:14.541651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:14.541651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:14.541651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:14.542534 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:14.542534 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:14.542534 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:14.542534 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:14.542534 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 348/689] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:45:14.548127 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826e5fa5-6ad0-4fb8-9acd-4b7467f59c04 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:14.555759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 342/690] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:45:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:14.561518 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a71773-af8c-4590-b8fd-0b65e23ef1e8 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:14.669289 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a71773-af8c-4590-b8fd-0b65e23ef1e8 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:14.673092 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826e5fa5-6ad0-4fb8-9acd-4b7467f59c04 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:14.714726 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8095ffe2-3cf4-45fe-8a7f-f7177ef3ce3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:14.715823 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2879a4a-2a93-4431-abd2-f8d3ca7d6dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:14.716155 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095ffe2-3cf4-45fe-8a7f-f7177ef3ce3a None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:14.716495 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2879a4a-2a93-4431-abd2-f8d3ca7d6dbf None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:14.716802 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2879a4a-2a93-4431-abd2-f8d3ca7d6dbf None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:14.717081 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2879a4a-2a93-4431-abd2-f8d3ca7d6dbf None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:14.717307 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095ffe2-3cf4-45fe-8a7f-f7177ef3ce3a None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:14.717764 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095ffe2-3cf4-45fe-8a7f-f7177ef3ce3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:14.717863 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2879a4a-2a93-4431-abd2-f8d3ca7d6dbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:14.718098 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2879a4a-2a93-4431-abd2-f8d3ca7d6dbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:14.719649 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2879a4a-2a93-4431-abd2-f8d3ca7d6dbf None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:14.732167 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8095ffe2-3cf4-45fe-8a7f-f7177ef3ce3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:14.733548 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 349/691] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:14.734276 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8095ffe2-3cf4-45fe-8a7f-f7177ef3ce3a None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:14.739502 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2879a4a-2a93-4431-abd2-f8d3ca7d6dbf None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:14.760099 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-8095ffe2-3cf4-45fe-8a7f-f7177ef3ce3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:14.760099 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:14.760099 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:14.760099 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:14.760099 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:14.760099 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:14.760099 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:14.760099 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:14.760099 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:14.760099 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:14.760099 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:14.760099 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:14.760099 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:14.760099 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:14.760099 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:14.761054 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:14.761054 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:14.761054 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:14.761054 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:14.761054 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:14.761054 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:14.761054 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:14.761054 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:14.761054 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:14.761054 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:14.761054 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:14.761054 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:14.761054 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:14.761054 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:14.761054 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:14.761054 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:14.761054 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:14.761961 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:14.761961 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:14.761961 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:14.761961 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:14.764747 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 343/692] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:14.770256 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8095ffe2-3cf4-45fe-8a7f-f7177ef3ce3a None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:14.877884 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8095ffe2-3cf4-45fe-8a7f-f7177ef3ce3a None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:14.890971 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2879a4a-2a93-4431-abd2-f8d3ca7d6dbf None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:14.920259 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51659904-76ca-45fa-9334-6b4beb7e0627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:14.921374 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51659904-76ca-45fa-9334-6b4beb7e0627 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:14.923223 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51659904-76ca-45fa-9334-6b4beb7e0627 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:14.923701 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51659904-76ca-45fa-9334-6b4beb7e0627 None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:14.924705 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51659904-76ca-45fa-9334-6b4beb7e0627 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:14.926148 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51659904-76ca-45fa-9334-6b4beb7e0627 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:14.928261 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51659904-76ca-45fa-9334-6b4beb7e0627 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:14.952995 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 344/693] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:14.961012 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51659904-76ca-45fa-9334-6b4beb7e0627 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:14.969528 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbad774-1088-4b80-8ea6-a699a74c3a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:14.970936 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbad774-1088-4b80-8ea6-a699a74c3a70 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:14.973165 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbad774-1088-4b80-8ea6-a699a74c3a70 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:14.973644 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbad774-1088-4b80-8ea6-a699a74c3a70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:14.987390 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbad774-1088-4b80-8ea6-a699a74c3a70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:14.988870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbad774-1088-4b80-8ea6-a699a74c3a70 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:15.020103 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-9bbad774-1088-4b80-8ea6-a699a74c3a70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:15.020103 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:15.020103 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:15.020103 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:15.020103 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:15.020103 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:15.020103 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:15.020103 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:15.020103 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:15.020103 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:15.020103 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:15.020103 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:15.020103 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 06 14:45:15.020103 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(project_id) May 06 14:45:15.020103 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 06 14:45:15.021596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 06 14:45:15.021596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:15.021596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:15.021596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:15.021596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:15.021596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:15.021596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:15.021596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:15.021596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:15.021596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:15.021596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:15.021596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:15.021596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:15.021596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:15.021596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:15.021596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:15.021596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:15.022951 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 06 14:45:15.022951 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 06 14:45:15.022951 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 06 14:45:15.022951 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 06 14:45:15.022951 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 06 14:45:15.022951 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 06 14:45:15.022951 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:15.022951 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:15.022951 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 350/694] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:45:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:45:15.028450 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbad774-1088-4b80-8ea6-a699a74c3a70 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:15.108444 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51659904-76ca-45fa-9334-6b4beb7e0627 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:15.148400 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbad774-1088-4b80-8ea6-a699a74c3a70 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:15.160891 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728aeccb-be50-456a-94e8-eba5f3eccaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:15.162358 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728aeccb-be50-456a-94e8-eba5f3eccaab None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:15.162702 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728aeccb-be50-456a-94e8-eba5f3eccaab None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:15.163030 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728aeccb-be50-456a-94e8-eba5f3eccaab None admin] PATH_INFO: `/v3/users/nova` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:15.170655 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728aeccb-be50-456a-94e8-eba5f3eccaab None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:15.172220 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728aeccb-be50-456a-94e8-eba5f3eccaab None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:15.186860 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-728aeccb-be50-456a-94e8-eba5f3eccaab None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 06 14:45:15.186860 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:15.186860 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:15.186860 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:15.186860 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:15.186860 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:15.186860 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:15.186860 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:15.186860 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:15.186860 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:15.186860 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:15.186860 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:15.186860 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 06 14:45:15.186860 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_user(user_id) May 06 14:45:15.186860 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 06 14:45:15.186860 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 06 14:45:15.187855 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:15.187855 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:15.187855 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 06 14:45:15.187855 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:15.187855 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 06 14:45:15.187855 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:15.187855 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:15.187855 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:15.187855 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:15.187855 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:15.187855 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:15.187855 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:15.187855 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:15.187855 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:15.187855 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:15.187855 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:15.187855 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:15.187855 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:15.188824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:15.188824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:15.188824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 06 14:45:15.188824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 06 14:45:15.188824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 06 14:45:15.188824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 06 14:45:15.188824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 06 14:45:15.188824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 06 14:45:15.188824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 06 14:45:15.188824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:15.188824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 345/695] 10.222.0.23 () {56 vars in 1093 bytes} [Fri May 6 14:45:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:15.193923 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845c488-a38f-4eb8-81d3-36ebd7f386aa None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:15.194213 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845c488-a38f-4eb8-81d3-36ebd7f386aa None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:15.194489 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845c488-a38f-4eb8-81d3-36ebd7f386aa None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:15.194904 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f5efb-f0c8-4ad5-aa66-92624c20efc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:15.195787 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f5efb-f0c8-4ad5-aa66-92624c20efc4 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:15.196262 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 346/696] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:45:15.196516 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f5efb-f0c8-4ad5-aa66-92624c20efc4 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:15.197007 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f5efb-f0c8-4ad5-aa66-92624c20efc4 None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:15.197534 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845c488-a38f-4eb8-81d3-36ebd7f386aa None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:15.198235 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987f5efb-f0c8-4ad5-aa66-92624c20efc4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:15.198877 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987f5efb-f0c8-4ad5-aa66-92624c20efc4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:15.200547 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987f5efb-f0c8-4ad5-aa66-92624c20efc4 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:15.217941 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 351/697] 10.222.0.23 () {56 vars in 1148 bytes} [Fri May 6 14:45:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:15.226564 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f5efb-f0c8-4ad5-aa66-92624c20efc4 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:15.315271 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845c488-a38f-4eb8-81d3-36ebd7f386aa None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:15.341896 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f5efb-f0c8-4ad5-aa66-92624c20efc4 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:15.361729 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e5782-05bc-4fa4-bfab-3b8df766e1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:15.362716 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e5782-05bc-4fa4-bfab-3b8df766e1ff None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:15.363003 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e5782-05bc-4fa4-bfab-3b8df766e1ff None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:15.363289 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e5782-05bc-4fa4-bfab-3b8df766e1ff None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:15.364136 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70e5782-05bc-4fa4-bfab-3b8df766e1ff None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:15.364450 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70e5782-05bc-4fa4-bfab-3b8df766e1ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:15.366910 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70e5782-05bc-4fa4-bfab-3b8df766e1ff None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:15.379165 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7157bc-5642-4259-a277-6a09dcfa4f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:15.379923 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7157bc-5642-4259-a277-6a09dcfa4f0b None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:15.380287 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7157bc-5642-4259-a277-6a09dcfa4f0b None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:15.381369 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7157bc-5642-4259-a277-6a09dcfa4f0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:15.382345 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7157bc-5642-4259-a277-6a09dcfa4f0b None admin] RBAC: Adding query filter params (role.id=f5641817ee56438496afc6135d37a4c0, scope.project.id=d5398feee6eb490cb49bb261448a1a9b, user.id=bf9eed62c8f1409282c2af6488426a0f) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:15.382725 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7157bc-5642-4259-a277-6a09dcfa4f0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:15.384232 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7157bc-5642-4259-a277-6a09dcfa4f0b None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:15.393937 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 347/698] 10.222.0.23 () {56 vars in 1133 bytes} [Fri May 6 14:45:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:15.395394 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e23cf12-a73b-4f64-8531-6db8e6ba3c05 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:15.395687 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e23cf12-a73b-4f64-8531-6db8e6ba3c05 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:15.395997 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e23cf12-a73b-4f64-8531-6db8e6ba3c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:15.397127 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 352/699] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:45:15 2022] GET /identity/v3/role_assignments?user.id=bf9eed62c8f1409282c2af6488426a0f&scope.project.id=d5398feee6eb490cb49bb261448a1a9b&role.id=f5641817ee56438496afc6135d37a4c0 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:15.398201 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7157bc-5642-4259-a277-6a09dcfa4f0b None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:15.416585 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-0e23cf12-a73b-4f64-8531-6db8e6ba3c05 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:15.417886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-0e23cf12-a73b-4f64-8531-6db8e6ba3c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:15.500686 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 348/700] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:45:15.545237 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e23cf12-a73b-4f64-8531-6db8e6ba3c05 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:15.670646 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7157bc-5642-4259-a277-6a09dcfa4f0b None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:15.680250 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e23cf12-a73b-4f64-8531-6db8e6ba3c05 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:15.705190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af7c003-4169-48ee-944f-9cdd618a2ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:15.705862 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af7c003-4169-48ee-944f-9cdd618a2ef2 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:15.706161 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af7c003-4169-48ee-944f-9cdd618a2ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:15.706402 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af7c003-4169-48ee-944f-9cdd618a2ef2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:15.723753 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef6d3d-09eb-45f0-b2ad-f9fff1d12489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:15.725638 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef6d3d-09eb-45f0-b2ad-f9fff1d12489 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:15.726109 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef6d3d-09eb-45f0-b2ad-f9fff1d12489 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:15.726578 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef6d3d-09eb-45f0-b2ad-f9fff1d12489 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:15.727725 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ef6d3d-09eb-45f0-b2ad-f9fff1d12489 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:15.730196 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ef6d3d-09eb-45f0-b2ad-f9fff1d12489 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:15.754644 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-76ef6d3d-09eb-45f0-b2ad-f9fff1d12489 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 06 14:45:15.754644 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:15.754644 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 06 14:45:15.754644 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 06 14:45:15.754644 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 06 14:45:15.754644 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 06 14:45:15.754644 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 06 14:45:15.754644 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:15.754644 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 06 14:45:15.754644 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:15.754644 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 06 14:45:15.754644 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:15.754644 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:15.754644 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:15.754644 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:15.754644 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:15.755895 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:15.755895 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:15.755895 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:15.755895 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:15.755895 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:15.755895 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:15.755895 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:15.755895 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 06 14:45:15.755895 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(service_id) May 06 14:45:15.755895 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 06 14:45:15.755895 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 06 14:45:15.755895 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:15.755895 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:15.755895 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:15.755895 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:15.755895 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:15.756762 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:15.756762 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:15.756762 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:15.756762 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:15.756762 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:15.756762 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:15.756762 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:15.756762 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:15.756762 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:15.756762 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:15.756762 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:15.756762 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 06 14:45:15.756762 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:15.756762 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 06 14:45:15.756762 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:15.757650 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 349/701] 10.222.0.23 () {56 vars in 1129 bytes} [Fri May 6 14:45:15 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:15.763388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef6d3d-09eb-45f0-b2ad-f9fff1d12489 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:15.775877 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af7c003-4169-48ee-944f-9cdd618a2ef2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:15.777365 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af7c003-4169-48ee-944f-9cdd618a2ef2 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:15.878879 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 353/702] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:45:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 06 14:45:15.882527 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef6d3d-09eb-45f0-b2ad-f9fff1d12489 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:15.884507 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af7c003-4169-48ee-944f-9cdd618a2ef2 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:15.930218 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b289f43-e74e-483e-bb71-2465678b77bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:15.931177 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b289f43-e74e-483e-bb71-2465678b77bd None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:15.931637 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b289f43-e74e-483e-bb71-2465678b77bd None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:15.932205 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b289f43-e74e-483e-bb71-2465678b77bd None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:15.933142 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b289f43-e74e-483e-bb71-2465678b77bd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:15.933555 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b289f43-e74e-483e-bb71-2465678b77bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:15.935078 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b289f43-e74e-483e-bb71-2465678b77bd None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:15.940834 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 350/703] 10.222.0.23 () {56 vars in 1124 bytes} [Fri May 6 14:45:15 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:15.948350 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b289f43-e74e-483e-bb71-2465678b77bd None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:15.987371 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af7c003-4169-48ee-944f-9cdd618a2ef2 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:16.023163 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda97d6-ea24-465b-8ec0-29b743450f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:16.023963 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda97d6-ea24-465b-8ec0-29b743450f54 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:16.024252 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda97d6-ea24-465b-8ec0-29b743450f54 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:16.024533 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda97d6-ea24-465b-8ec0-29b743450f54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:16.035455 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eda97d6-ea24-465b-8ec0-29b743450f54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:16.037539 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eda97d6-ea24-465b-8ec0-29b743450f54 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:16.055999 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-5eda97d6-ea24-465b-8ec0-29b743450f54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:16.055999 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:16.055999 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:16.055999 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:16.055999 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:16.055999 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:16.055999 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:16.055999 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:16.055999 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:16.055999 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:16.055999 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:16.055999 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:16.055999 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:16.055999 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:16.055999 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:16.057304 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:16.057304 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:16.057304 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:16.057304 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:16.057304 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:16.057304 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:16.057304 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:16.057304 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:16.057304 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:16.057304 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:16.057304 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:16.057304 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:16.057304 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:16.057304 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:16.057304 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:16.057304 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:16.057304 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:16.058318 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:16.058318 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:16.058318 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:16.058318 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:16.058318 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 354/704] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:45:16.062992 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda97d6-ea24-465b-8ec0-29b743450f54 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:16.084838 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b289f43-e74e-483e-bb71-2465678b77bd None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:16.119212 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2416b8d-286b-4c5f-a632-55a92a81c277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:16.119889 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2416b8d-286b-4c5f-a632-55a92a81c277 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:16.120175 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2416b8d-286b-4c5f-a632-55a92a81c277 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:16.120454 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2416b8d-286b-4c5f-a632-55a92a81c277 None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:16.121205 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2416b8d-286b-4c5f-a632-55a92a81c277 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:16.121504 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2416b8d-286b-4c5f-a632-55a92a81c277 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:16.122885 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2416b8d-286b-4c5f-a632-55a92a81c277 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:16.129184 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 351/705] 10.222.0.23 () {56 vars in 1124 bytes} [Fri May 6 14:45:15 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:16.182874 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda97d6-ea24-465b-8ec0-29b743450f54 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:16.232450 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4439053-ec5b-42ee-9929-25b3b810ef57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:16.233181 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4439053-ec5b-42ee-9929-25b3b810ef57 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:16.233469 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4439053-ec5b-42ee-9929-25b3b810ef57 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:16.233750 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4439053-ec5b-42ee-9929-25b3b810ef57 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:16.234499 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4439053-ec5b-42ee-9929-25b3b810ef57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:16.234807 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4439053-ec5b-42ee-9929-25b3b810ef57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:16.236375 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4439053-ec5b-42ee-9929-25b3b810ef57 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:16.249770 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 355/706] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:16.254855 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2416b8d-286b-4c5f-a632-55a92a81c277 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:16.373880 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2416b8d-286b-4c5f-a632-55a92a81c277 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:16.423353 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4d93c4-32f7-40b5-a4fb-8737c8be0f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:16.424212 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d93c4-32f7-40b5-a4fb-8737c8be0f92 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:16.424512 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d93c4-32f7-40b5-a4fb-8737c8be0f92 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:16.424995 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d93c4-32f7-40b5-a4fb-8737c8be0f92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:16.434122 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4d93c4-32f7-40b5-a4fb-8737c8be0f92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:16.435508 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4d93c4-32f7-40b5-a4fb-8737c8be0f92 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:16.455694 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-5b4d93c4-32f7-40b5-a4fb-8737c8be0f92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:16.455694 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:16.455694 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:16.455694 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:16.455694 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:16.455694 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:16.455694 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:16.455694 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:16.455694 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:16.455694 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:16.455694 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:16.455694 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:16.455694 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 06 14:45:16.455694 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(project_id) May 06 14:45:16.455694 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 06 14:45:16.457869 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 06 14:45:16.457869 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:16.457869 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:16.457869 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:16.457869 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:16.457869 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:16.457869 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:16.457869 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:16.457869 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:16.457869 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:16.457869 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:16.457869 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:16.457869 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:16.457869 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:16.457869 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:16.457869 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:16.457869 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:16.460693 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 06 14:45:16.460693 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 06 14:45:16.460693 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 06 14:45:16.460693 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 06 14:45:16.460693 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 06 14:45:16.460693 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 06 14:45:16.460693 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:16.460693 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:16.460693 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 352/707] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:45:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:16.464626 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4439053-ec5b-42ee-9929-25b3b810ef57 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:16.567805 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4439053-ec5b-42ee-9929-25b3b810ef57 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:16.606180 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f01c97-f03d-4415-998c-1c9f43b08458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:16.606844 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f01c97-f03d-4415-998c-1c9f43b08458 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:16.607124 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f01c97-f03d-4415-998c-1c9f43b08458 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:16.607382 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f01c97-f03d-4415-998c-1c9f43b08458 None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:16.608167 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f01c97-f03d-4415-998c-1c9f43b08458 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:16.608465 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f01c97-f03d-4415-998c-1c9f43b08458 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:16.609927 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f01c97-f03d-4415-998c-1c9f43b08458 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:16.625734 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574c63e5-484c-4ab4-9a70-2cbfc5f77754 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:16.625734 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574c63e5-484c-4ab4-9a70-2cbfc5f77754 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:16.625734 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574c63e5-484c-4ab4-9a70-2cbfc5f77754 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:16.626651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 356/708] 10.222.0.23 () {56 vars in 1148 bytes} [Fri May 6 14:45:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:16.631088 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 353/709] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:45:16.635832 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f01c97-f03d-4415-998c-1c9f43b08458 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:16.636347 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefca81f-f241-4151-82f1-e889c2b9c6fb None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:16.636704 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefca81f-f241-4151-82f1-e889c2b9c6fb None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:16.636982 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefca81f-f241-4151-82f1-e889c2b9c6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:16.678391 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-aefca81f-f241-4151-82f1-e889c2b9c6fb None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:16.679607 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-aefca81f-f241-4151-82f1-e889c2b9c6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:16.762858 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f01c97-f03d-4415-998c-1c9f43b08458 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:16.804173 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a52af9-2336-45d4-b4cc-f8f21fd5fdd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:16.805369 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a52af9-2336-45d4-b4cc-f8f21fd5fdd8 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:16.805883 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a52af9-2336-45d4-b4cc-f8f21fd5fdd8 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:16.806382 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a52af9-2336-45d4-b4cc-f8f21fd5fdd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:16.806785 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 354/710] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 06 14:45:16.808860 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a52af9-2336-45d4-b4cc-f8f21fd5fdd8 None admin] RBAC: Adding query filter params (role.id=6dcd35d1a83748759d6ef9f026077835, scope.project.id=d5398feee6eb490cb49bb261448a1a9b, user.id=fc8cf09642ec4e10a80badcf8b585ef8) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:16.809431 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a52af9-2336-45d4-b4cc-f8f21fd5fdd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:16.812007 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a52af9-2336-45d4-b4cc-f8f21fd5fdd8 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:16.828146 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 357/711] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:45:16 2022] GET /identity/v3/role_assignments?user.id=fc8cf09642ec4e10a80badcf8b585ef8&scope.project.id=d5398feee6eb490cb49bb261448a1a9b&role.id=6dcd35d1a83748759d6ef9f026077835 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:16.832317 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefca81f-f241-4151-82f1-e889c2b9c6fb None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:16.946119 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefca81f-f241-4151-82f1-e889c2b9c6fb None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:16.998896 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6756abcb-2c53-434c-8238-00e8a6c51c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:16.999709 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6756abcb-2c53-434c-8238-00e8a6c51c7b None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:16.999989 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6756abcb-2c53-434c-8238-00e8a6c51c7b None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:17.000261 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6756abcb-2c53-434c-8238-00e8a6c51c7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:17.015131 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6756abcb-2c53-434c-8238-00e8a6c51c7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:17.016573 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6756abcb-2c53-434c-8238-00e8a6c51c7b None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:17.036753 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-6756abcb-2c53-434c-8238-00e8a6c51c7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:17.036753 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:17.036753 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:17.036753 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:17.036753 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:17.036753 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:17.036753 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:17.036753 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:17.036753 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:17.036753 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:17.036753 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:17.036753 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:17.036753 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:17.036753 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:17.036753 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:17.038039 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:17.038039 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:17.038039 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:17.038039 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:17.038039 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:17.038039 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:17.038039 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:17.038039 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:17.038039 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:17.038039 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:17.038039 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:17.038039 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:17.038039 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:17.038039 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:17.038039 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:17.038039 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:17.038039 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:17.039013 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:17.039013 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:17.039013 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:17.039013 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:17.039013 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 355/712] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:17.044457 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a52af9-2336-45d4-b4cc-f8f21fd5fdd8 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:17.161634 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a52af9-2336-45d4-b4cc-f8f21fd5fdd8 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:17.201176 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e57ba8-3b58-4f66-b85e-e40d8d38a1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:17.202203 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e57ba8-3b58-4f66-b85e-e40d8d38a1bb None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:17.202683 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e57ba8-3b58-4f66-b85e-e40d8d38a1bb None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:17.203179 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e57ba8-3b58-4f66-b85e-e40d8d38a1bb None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:17.204100 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e57ba8-3b58-4f66-b85e-e40d8d38a1bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:17.204616 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e57ba8-3b58-4f66-b85e-e40d8d38a1bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:17.206179 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e57ba8-3b58-4f66-b85e-e40d8d38a1bb None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:17.219447 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 358/713] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:17.224957 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6756abcb-2c53-434c-8238-00e8a6c51c7b None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:17.379357 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d302f8-47d4-4e6d-b917-95afd739ca79 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:17.379967 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d302f8-47d4-4e6d-b917-95afd739ca79 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:17.381064 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d302f8-47d4-4e6d-b917-95afd739ca79 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:17.383333 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 359/714] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:45:17.387550 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6756abcb-2c53-434c-8238-00e8a6c51c7b None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:17.389128 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed64869-7506-4c5f-9253-9ca937e87977 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:17.389703 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed64869-7506-4c5f-9253-9ca937e87977 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:17.390185 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed64869-7506-4c5f-9253-9ca937e87977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:17.423445 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-7ed64869-7506-4c5f-9253-9ca937e87977 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:17.424692 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-7ed64869-7506-4c5f-9253-9ca937e87977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:17.425935 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf894af-827e-4af6-9a19-f2c40e8927d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:17.427629 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf894af-827e-4af6-9a19-f2c40e8927d4 None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:17.428127 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf894af-827e-4af6-9a19-f2c40e8927d4 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:17.428631 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf894af-827e-4af6-9a19-f2c40e8927d4 None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:17.429563 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf894af-827e-4af6-9a19-f2c40e8927d4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:17.431230 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf894af-827e-4af6-9a19-f2c40e8927d4 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:17.458000 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 356/715] 10.222.0.23 () {60 vars in 1132 bytes} [Fri May 6 14:45:17 2022] POST /identity/v3/users => generated 254 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:45:17.531987 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 360/716] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 06 14:45:17.557897 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf894af-827e-4af6-9a19-f2c40e8927d4 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:17.687674 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf894af-827e-4af6-9a19-f2c40e8927d4 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:17.731144 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb884b2-c637-488b-8754-1f2b22fe6056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:17.732354 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb884b2-c637-488b-8754-1f2b22fe6056 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:17.732868 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb884b2-c637-488b-8754-1f2b22fe6056 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:17.733335 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb884b2-c637-488b-8754-1f2b22fe6056 None admin] PATH_INFO: `/v3/services/network` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:17.734178 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb884b2-c637-488b-8754-1f2b22fe6056 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:17.735750 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb884b2-c637-488b-8754-1f2b22fe6056 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:17.748744 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-1cb884b2-c637-488b-8754-1f2b22fe6056 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 06 14:45:17.748744 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:17.748744 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 06 14:45:17.748744 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 06 14:45:17.748744 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 06 14:45:17.748744 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 06 14:45:17.748744 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 06 14:45:17.748744 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:17.748744 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 06 14:45:17.748744 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:17.748744 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 06 14:45:17.748744 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:17.748744 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:17.748744 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:17.748744 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:17.748744 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:17.748744 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:17.749733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:17.749733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:17.749733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:17.749733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:17.749733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:17.749733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:17.749733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 06 14:45:17.749733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(service_id) May 06 14:45:17.749733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 06 14:45:17.749733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 06 14:45:17.749733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:17.749733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:17.749733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:17.749733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:17.749733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:17.749733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:17.750580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:17.750580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:17.750580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:17.750580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:17.750580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:17.750580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:17.750580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:17.750580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:17.750580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:17.750580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:17.750580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 06 14:45:17.750580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:17.750580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 06 14:45:17.750580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:17.753437 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 357/717] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:45:17 2022] GET /identity/v3/services/network => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:17.758983 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed64869-7506-4c5f-9253-9ca937e87977 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:17.866426 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed64869-7506-4c5f-9253-9ca937e87977 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:17.904228 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6c6611-0c3d-44d8-866f-746a05c26d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:17.904938 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c6611-0c3d-44d8-866f-746a05c26d3d None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:17.905223 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c6611-0c3d-44d8-866f-746a05c26d3d None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:17.905751 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c6611-0c3d-44d8-866f-746a05c26d3d None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:17.906467 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6c6611-0c3d-44d8-866f-746a05c26d3d None admin] RBAC: Adding query filter params (name=network) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:17.906778 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6c6611-0c3d-44d8-866f-746a05c26d3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:17.908158 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6c6611-0c3d-44d8-866f-746a05c26d3d None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:17.914069 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 361/718] 10.222.0.23 () {56 vars in 1112 bytes} [Fri May 6 14:45:17 2022] GET /identity/v3/services?name=network => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:17.919133 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb884b2-c637-488b-8754-1f2b22fe6056 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:18.033212 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb884b2-c637-488b-8754-1f2b22fe6056 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:18.070179 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc3b6c2-56b9-4947-a602-4b554bff91f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:18.071203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc3b6c2-56b9-4947-a602-4b554bff91f9 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:18.071719 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc3b6c2-56b9-4947-a602-4b554bff91f9 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:18.073815 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc3b6c2-56b9-4947-a602-4b554bff91f9 None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:18.074695 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc3b6c2-56b9-4947-a602-4b554bff91f9 None admin] RBAC: Adding query filter params (type=network) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:18.075155 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc3b6c2-56b9-4947-a602-4b554bff91f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:18.076738 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc3b6c2-56b9-4947-a602-4b554bff91f9 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:18.081785 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 358/719] 10.222.0.23 () {56 vars in 1112 bytes} [Fri May 6 14:45:17 2022] GET /identity/v3/services?type=network => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:18.611781 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a693ec5-3ce1-4532-b932-ce9e9676f609 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:18.612293 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a693ec5-3ce1-4532-b932-ce9e9676f609 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:18.612369 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a693ec5-3ce1-4532-b932-ce9e9676f609 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:18.614005 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 362/720] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:45:18.620475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264caeb0-d06c-4631-9c9a-550daa4316c8 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:18.620795 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264caeb0-d06c-4631-9c9a-550daa4316c8 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:18.621074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264caeb0-d06c-4631-9c9a-550daa4316c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:18.645580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-264caeb0-d06c-4631-9c9a-550daa4316c8 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:18.646817 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-264caeb0-d06c-4631-9c9a-550daa4316c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:18.738240 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 359/721] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:45:18.786076 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a693ec5-3ce1-4532-b932-ce9e9676f609 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:18.942353 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a693ec5-3ce1-4532-b932-ce9e9676f609 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:18.989333 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7ed08-eee7-45ed-8e36-621ce50250b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:18.990742 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7ed08-eee7-45ed-8e36-621ce50250b4 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:18.991017 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7ed08-eee7-45ed-8e36-621ce50250b4 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:18.991288 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7ed08-eee7-45ed-8e36-621ce50250b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:19.013491 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd7ed08-eee7-45ed-8e36-621ce50250b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:45:19.021104 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-8fd7ed08-eee7-45ed-8e36-621ce50250b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:45:19.021104 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:19.021104 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:19.021104 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:19.021104 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:19.021104 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:19.021104 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:19.021104 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:19.021104 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:19.021104 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:19.021104 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:19.021104 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:19.021104 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:45:19.021104 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:45:19.021104 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:45:19.021104 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:45:19.022363 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:45:19.022363 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:45:19.022363 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:45:19.022363 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:45:19.022363 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:19.022363 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:19.022363 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:19.022363 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:19.022363 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:19.022363 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:19.022363 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:19.022363 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:19.022363 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:19.022363 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:19.022363 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:19.022363 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:19.023188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:19.023188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:19.023188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:19.023188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:19.023188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:45:19.023188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:45:19.023188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:45:19.023188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:45:19.023188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:45:19.023188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:45:19.023188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:45:19.023188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:19.023829 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 363/722] 10.222.0.23 () {56 vars in 1096 bytes} [Fri May 6 14:45:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:19.028440 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264caeb0-d06c-4631-9c9a-550daa4316c8 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:19.139282 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463a7eb-683a-44e2-8073-7075df8fe665 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:19.139907 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463a7eb-683a-44e2-8073-7075df8fe665 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:19.140445 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463a7eb-683a-44e2-8073-7075df8fe665 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:19.142394 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 364/723] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:45:19.149966 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b0a510-3227-4165-8c7a-6674dc58856b None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:19.150515 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b0a510-3227-4165-8c7a-6674dc58856b None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:19.151007 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b0a510-3227-4165-8c7a-6674dc58856b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:19.163569 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264caeb0-d06c-4631-9c9a-550daa4316c8 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:19.192296 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-e0b0a510-3227-4165-8c7a-6674dc58856b None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:19.194796 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e0b0a510-3227-4165-8c7a-6674dc58856b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:19.204729 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3457b1e6-b3f6-43a4-8825-b7de5c798f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:19.205427 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3457b1e6-b3f6-43a4-8825-b7de5c798f85 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:19.205711 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3457b1e6-b3f6-43a4-8825-b7de5c798f85 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:19.205983 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3457b1e6-b3f6-43a4-8825-b7de5c798f85 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:19.206735 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3457b1e6-b3f6-43a4-8825-b7de5c798f85 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:19.207042 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3457b1e6-b3f6-43a4-8825-b7de5c798f85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:19.209129 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3457b1e6-b3f6-43a4-8825-b7de5c798f85 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:19.224502 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 360/724] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:45:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:19.230040 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3457b1e6-b3f6-43a4-8825-b7de5c798f85 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:19.315256 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 365/725] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:45:19.349894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b0a510-3227-4165-8c7a-6674dc58856b None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:19.351554 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3457b1e6-b3f6-43a4-8825-b7de5c798f85 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:19.388790 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62626a8-4f43-4bd3-a70b-d1dd52ae41ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:19.389948 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62626a8-4f43-4bd3-a70b-d1dd52ae41ba None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:19.390423 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62626a8-4f43-4bd3-a70b-d1dd52ae41ba None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:19.390948 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62626a8-4f43-4bd3-a70b-d1dd52ae41ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:19.402809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62626a8-4f43-4bd3-a70b-d1dd52ae41ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:19.404575 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62626a8-4f43-4bd3-a70b-d1dd52ae41ba None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:19.422545 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-e62626a8-4f43-4bd3-a70b-d1dd52ae41ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:19.422545 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:19.422545 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:19.422545 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:19.422545 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:19.422545 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:19.422545 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:19.422545 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:19.422545 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:19.422545 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:19.422545 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:19.422545 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:19.422545 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:19.422545 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:19.422545 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:19.423468 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:19.423468 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:19.423468 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:19.423468 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:19.423468 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:19.423468 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:19.423468 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:19.423468 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:19.423468 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:19.423468 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:19.423468 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:19.423468 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:19.423468 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:19.423468 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:19.423468 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:19.423468 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:19.423468 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:19.424350 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:19.424350 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:19.424350 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:19.424350 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:19.426561 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 361/726] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:45:19.433427 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62626a8-4f43-4bd3-a70b-d1dd52ae41ba None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:19.556347 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b0a510-3227-4165-8c7a-6674dc58856b None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:19.572629 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62626a8-4f43-4bd3-a70b-d1dd52ae41ba None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:19.606595 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb8ede2-47d7-441c-b543-8060f1c74dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:19.607264 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb8ede2-47d7-441c-b543-8060f1c74dcd None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:19.607575 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb8ede2-47d7-441c-b543-8060f1c74dcd None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:19.607821 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb8ede2-47d7-441c-b543-8060f1c74dcd None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:19.608551 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb8ede2-47d7-441c-b543-8060f1c74dcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:19.608880 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb8ede2-47d7-441c-b543-8060f1c74dcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:19.610241 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb8ede2-47d7-441c-b543-8060f1c74dcd None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:19.626294 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 362/727] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:19.633305 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb8ede2-47d7-441c-b543-8060f1c74dcd None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:19.633748 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee79d3-0b13-4720-9aea-8b067f1bf41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:19.634428 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee79d3-0b13-4720-9aea-8b067f1bf41b None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:19.634711 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee79d3-0b13-4720-9aea-8b067f1bf41b None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:19.634989 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee79d3-0b13-4720-9aea-8b067f1bf41b None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:19.635705 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cee79d3-0b13-4720-9aea-8b067f1bf41b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:19.637108 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cee79d3-0b13-4720-9aea-8b067f1bf41b None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:19.659211 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 366/728] 10.222.0.23 () {60 vars in 1141 bytes} [Fri May 6 14:45:19 2022] POST /identity/v3/services => generated 251 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 14:45:19.733335 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51d742-87cb-47ce-acc0-5da97bd22450 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:19.734162 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51d742-87cb-47ce-acc0-5da97bd22450 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:19.734649 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51d742-87cb-47ce-acc0-5da97bd22450 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:19.736513 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 367/729] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:45:19.742441 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97c393-447f-4420-8d9f-8c63d54d32a7 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:19.743081 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97c393-447f-4420-8d9f-8c63d54d32a7 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:19.743572 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97c393-447f-4420-8d9f-8c63d54d32a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:19.779768 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-eb97c393-447f-4420-8d9f-8c63d54d32a7 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:19.781411 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-eb97c393-447f-4420-8d9f-8c63d54d32a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:19.803419 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb8ede2-47d7-441c-b543-8060f1c74dcd None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:19.870965 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e50b4c-60de-417d-abb2-078782c51647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:19.871696 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e50b4c-60de-417d-abb2-078782c51647 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:19.872396 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e50b4c-60de-417d-abb2-078782c51647 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:19.872700 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e50b4c-60de-417d-abb2-078782c51647 None admin] PATH_INFO: `/v3/users/swift` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:19.885383 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e50b4c-60de-417d-abb2-078782c51647 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:19.887618 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e50b4c-60de-417d-abb2-078782c51647 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:19.902582 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 368/730] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:19 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 06 14:45:19.906281 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-62e50b4c-60de-417d-abb2-078782c51647 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 06 14:45:19.906281 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:19.906281 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:19.906281 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:19.906281 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:19.906281 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:19.906281 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:19.906281 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:19.906281 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:19.906281 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:19.906281 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:19.906281 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:19.906281 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 06 14:45:19.906281 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_user(user_id) May 06 14:45:19.906281 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 06 14:45:19.906281 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 06 14:45:19.907171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:19.907171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:19.907171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 06 14:45:19.907171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:19.907171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 06 14:45:19.907171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:19.907171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:19.907171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:19.907171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:19.907171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:19.907171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:19.907171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:19.907171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:19.907171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:19.907171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:19.907171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:19.907171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:19.907171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:19.908095 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:19.908095 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:19.908095 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 06 14:45:19.908095 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 06 14:45:19.908095 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 06 14:45:19.908095 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 06 14:45:19.908095 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 06 14:45:19.908095 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 06 14:45:19.908095 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 06 14:45:19.908095 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:19.912206 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 363/731] 10.222.0.23 () {56 vars in 1096 bytes} [Fri May 6 14:45:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:19.913969 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb97c393-447f-4420-8d9f-8c63d54d32a7 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:19.928707 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e50b4c-60de-417d-abb2-078782c51647 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:20.026257 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb97c393-447f-4420-8d9f-8c63d54d32a7 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:20.055156 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e50b4c-60de-417d-abb2-078782c51647 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:20.062127 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be803a0d-3c1f-45d9-b52c-831a83126226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:20.062836 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be803a0d-3c1f-45d9-b52c-831a83126226 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:20.063292 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be803a0d-3c1f-45d9-b52c-831a83126226 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:20.063719 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be803a0d-3c1f-45d9-b52c-831a83126226 None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:20.064654 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be803a0d-3c1f-45d9-b52c-831a83126226 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:20.064968 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be803a0d-3c1f-45d9-b52c-831a83126226 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:20.066400 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be803a0d-3c1f-45d9-b52c-831a83126226 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:20.089069 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 369/732] 10.222.0.23 () {56 vars in 1135 bytes} [Fri May 6 14:45:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:20.093153 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be803a0d-3c1f-45d9-b52c-831a83126226 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:20.096931 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d507d6ae-f5aa-4c8f-98b1-3b64e69fad10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:20.097248 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d507d6ae-f5aa-4c8f-98b1-3b64e69fad10 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:20.097526 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d507d6ae-f5aa-4c8f-98b1-3b64e69fad10 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:20.097800 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d507d6ae-f5aa-4c8f-98b1-3b64e69fad10 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:20.110741 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d507d6ae-f5aa-4c8f-98b1-3b64e69fad10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:45:20.121229 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-d507d6ae-f5aa-4c8f-98b1-3b64e69fad10 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:45:20.121229 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:20.121229 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:20.121229 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:20.121229 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:20.121229 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:20.121229 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:20.121229 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:20.121229 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:20.121229 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:20.121229 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:20.121229 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:20.121229 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:45:20.121229 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:45:20.121229 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:45:20.121229 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:45:20.122150 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:45:20.122150 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:45:20.122150 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:45:20.122150 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:45:20.122150 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:20.122150 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:20.122150 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:20.122150 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:20.122150 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:20.122150 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:20.122150 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:20.122150 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:20.122150 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:20.122150 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:20.122150 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:20.122150 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:20.123472 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:20.123472 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:20.123472 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:20.123472 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:20.123472 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:45:20.123472 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:45:20.123472 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:45:20.123472 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:45:20.123472 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:45:20.123472 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:45:20.123472 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:45:20.123472 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:20.123472 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 364/733] 10.222.0.23 () {56 vars in 1096 bytes} [Fri May 6 14:45:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:20.129235 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d507d6ae-f5aa-4c8f-98b1-3b64e69fad10 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:20.273462 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be803a0d-3c1f-45d9-b52c-831a83126226 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:20.304749 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d507d6ae-f5aa-4c8f-98b1-3b64e69fad10 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:20.343361 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e104cf8e-04e6-4175-a676-8c20ab8e3c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:20.344035 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104cf8e-04e6-4175-a676-8c20ab8e3c17 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:20.344318 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104cf8e-04e6-4175-a676-8c20ab8e3c17 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:20.344594 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104cf8e-04e6-4175-a676-8c20ab8e3c17 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:20.345359 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e104cf8e-04e6-4175-a676-8c20ab8e3c17 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:20.345665 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e104cf8e-04e6-4175-a676-8c20ab8e3c17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:20.347035 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e104cf8e-04e6-4175-a676-8c20ab8e3c17 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:20.349202 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab5121b-7fbb-478b-be7e-930130c264f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:20.349862 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab5121b-7fbb-478b-be7e-930130c264f6 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:20.350138 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab5121b-7fbb-478b-be7e-930130c264f6 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:20.350412 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab5121b-7fbb-478b-be7e-930130c264f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:20.369040 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 365/734] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:45:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:20.375002 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e104cf8e-04e6-4175-a676-8c20ab8e3c17 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:20.472767 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab5121b-7fbb-478b-be7e-930130c264f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:20.474252 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab5121b-7fbb-478b-be7e-930130c264f6 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:20.509768 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e104cf8e-04e6-4175-a676-8c20ab8e3c17 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:20.548103 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609e5e23-7173-45fc-b365-52c380b86599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:20.550178 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e5e23-7173-45fc-b365-52c380b86599 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:20.550466 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e5e23-7173-45fc-b365-52c380b86599 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:20.550741 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e5e23-7173-45fc-b365-52c380b86599 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:20.570313 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609e5e23-7173-45fc-b365-52c380b86599 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:20.571790 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609e5e23-7173-45fc-b365-52c380b86599 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:20.576378 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 370/735] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:45:20 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 06 14:45:20.577780 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16250ef-0f49-46c4-aae1-d681d9dc1a65 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:20.578065 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16250ef-0f49-46c4-aae1-d681d9dc1a65 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:20.578341 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16250ef-0f49-46c4-aae1-d681d9dc1a65 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:20.579873 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 371/736] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:45:20.585700 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9bac44-7720-4637-bf91-9ead28a1f25d None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:20.586045 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9bac44-7720-4637-bf91-9ead28a1f25d None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:20.586256 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9bac44-7720-4637-bf91-9ead28a1f25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:20.596287 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-609e5e23-7173-45fc-b365-52c380b86599 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:20.596287 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:20.596287 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:20.596287 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:20.596287 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:20.596287 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:20.596287 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:20.596287 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:20.596287 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:20.596287 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:20.596287 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:20.596287 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:20.596287 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:20.596287 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:20.596287 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:20.598013 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:20.598013 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:20.598013 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:20.598013 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:20.598013 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:20.598013 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:20.598013 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:20.598013 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:20.598013 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:20.598013 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:20.598013 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:20.598013 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:20.598013 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:20.598013 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:20.598013 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:20.598013 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:20.598013 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:20.599020 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:20.599020 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:20.599020 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:20.599020 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:20.599020 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 366/737] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:20.599020 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609e5e23-7173-45fc-b365-52c380b86599 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:20.608077 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-8d9bac44-7720-4637-bf91-9ead28a1f25d None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:20.609314 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-8d9bac44-7720-4637-bf91-9ead28a1f25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:20.689056 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 372/738] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:20 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 06 14:45:20.689943 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9bac44-7720-4637-bf91-9ead28a1f25d None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:20.713378 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609e5e23-7173-45fc-b365-52c380b86599 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:20.753272 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e83a3c9-2bb3-4b05-8a58-90c9aeaade95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:20.754564 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83a3c9-2bb3-4b05-8a58-90c9aeaade95 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:20.755144 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83a3c9-2bb3-4b05-8a58-90c9aeaade95 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:20.755573 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83a3c9-2bb3-4b05-8a58-90c9aeaade95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:20.768826 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e83a3c9-2bb3-4b05-8a58-90c9aeaade95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:20.770299 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e83a3c9-2bb3-4b05-8a58-90c9aeaade95 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:20.789235 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-0e83a3c9-2bb3-4b05-8a58-90c9aeaade95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:20.789235 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:20.789235 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:20.789235 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:20.789235 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:20.789235 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:20.789235 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:20.789235 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:20.789235 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:20.789235 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:20.789235 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:20.789235 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:20.789235 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:20.789235 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:20.789235 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:20.791406 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:20.791406 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:20.791406 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:20.791406 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:20.791406 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:20.791406 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:20.791406 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:20.791406 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:20.791406 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:20.791406 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:20.791406 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:20.791406 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:20.791406 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:20.791406 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:20.791406 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:20.791406 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:20.791406 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:20.793730 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:20.793730 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:20.793730 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:20.793730 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:20.793730 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 367/739] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:20.793730 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e83a3c9-2bb3-4b05-8a58-90c9aeaade95 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:20.814443 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9bac44-7720-4637-bf91-9ead28a1f25d None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:20.850455 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a394270-c4c5-4aa7-adfe-fa3162313208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:20.851115 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a394270-c4c5-4aa7-adfe-fa3162313208 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:20.851387 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a394270-c4c5-4aa7-adfe-fa3162313208 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:20.851666 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a394270-c4c5-4aa7-adfe-fa3162313208 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:20.852399 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a394270-c4c5-4aa7-adfe-fa3162313208 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:20.852733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a394270-c4c5-4aa7-adfe-fa3162313208 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:20.854118 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a394270-c4c5-4aa7-adfe-fa3162313208 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:20.874112 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 373/740] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:20.875571 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a394270-c4c5-4aa7-adfe-fa3162313208 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:20.916243 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e83a3c9-2bb3-4b05-8a58-90c9aeaade95 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:20.957491 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e531d16-bacc-4096-88d0-cbf401d42938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:20.958979 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e531d16-bacc-4096-88d0-cbf401d42938 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:20.959208 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e531d16-bacc-4096-88d0-cbf401d42938 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:20.959644 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e531d16-bacc-4096-88d0-cbf401d42938 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:20.975607 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e531d16-bacc-4096-88d0-cbf401d42938 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:45:20.983954 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-9e531d16-bacc-4096-88d0-cbf401d42938 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 06 14:45:20.983954 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:20.983954 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:20.983954 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:20.983954 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:20.983954 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:20.983954 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:20.983954 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:20.983954 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:20.983954 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:20.983954 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:20.983954 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:20.983954 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:45:20.983954 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:45:20.983954 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:45:20.983954 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:45:20.986244 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:45:20.986244 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:45:20.986244 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:45:20.986244 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:45:20.986244 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:20.986244 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:20.986244 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:20.986244 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:20.986244 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:20.986244 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:20.986244 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:20.986244 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:20.986244 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:20.986244 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:20.986244 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:20.986244 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:20.988268 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:20.988268 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:20.988268 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:20.988268 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:20.988268 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:45:20.988268 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:45:20.988268 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:45:20.988268 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:45:20.988268 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:45:20.988268 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:45:20.988268 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 06 14:45:20.988268 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:20.990496 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a394270-c4c5-4aa7-adfe-fa3162313208 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:20.996670 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 368/741] 10.222.0.23 () {56 vars in 1102 bytes} [Fri May 6 14:45:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:20.996670 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e531d16-bacc-4096-88d0-cbf401d42938 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:21.023260 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9752c5a-7bac-4ade-8391-1a2928795177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:21.023923 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9752c5a-7bac-4ade-8391-1a2928795177 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:21.024209 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9752c5a-7bac-4ade-8391-1a2928795177 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:21.024485 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9752c5a-7bac-4ade-8391-1a2928795177 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:21.025249 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9752c5a-7bac-4ade-8391-1a2928795177 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:21.025553 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9752c5a-7bac-4ade-8391-1a2928795177 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:21.026939 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9752c5a-7bac-4ade-8391-1a2928795177 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:21.040135 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 374/742] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:21.045168 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9752c5a-7bac-4ade-8391-1a2928795177 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:21.106374 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e531d16-bacc-4096-88d0-cbf401d42938 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:21.146991 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9752c5a-7bac-4ade-8391-1a2928795177 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:21.149642 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e69528-2197-4e3b-a757-11bcb3a95835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:21.151284 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e69528-2197-4e3b-a757-11bcb3a95835 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:21.157808 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e69528-2197-4e3b-a757-11bcb3a95835 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:21.160770 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e69528-2197-4e3b-a757-11bcb3a95835 None admin] PATH_INFO: `/v3/users/nova` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:21.170007 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e69528-2197-4e3b-a757-11bcb3a95835 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:21.171467 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e69528-2197-4e3b-a757-11bcb3a95835 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:21.183765 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e658d17-06cc-44db-acd3-e9403b4a61a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:21.184436 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e658d17-06cc-44db-acd3-e9403b4a61a5 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:21.184746 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e658d17-06cc-44db-acd3-e9403b4a61a5 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:21.185024 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e658d17-06cc-44db-acd3-e9403b4a61a5 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:21.186596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e658d17-06cc-44db-acd3-e9403b4a61a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:21.186596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e658d17-06cc-44db-acd3-e9403b4a61a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:21.187416 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e658d17-06cc-44db-acd3-e9403b4a61a5 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:21.190888 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-04e69528-2197-4e3b-a757-11bcb3a95835 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 06 14:45:21.190888 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:21.190888 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:21.190888 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:21.190888 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:21.190888 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:21.190888 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:21.190888 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:21.190888 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:21.190888 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:21.190888 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:21.190888 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:21.190888 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 06 14:45:21.190888 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_user(user_id) May 06 14:45:21.190888 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 06 14:45:21.190888 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 06 14:45:21.192020 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:21.192020 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:21.192020 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 06 14:45:21.192020 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:21.192020 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 06 14:45:21.192020 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:21.192020 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:21.192020 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:21.192020 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:21.192020 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:21.192020 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:21.192020 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:21.192020 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:21.192020 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:21.192020 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:21.192020 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:21.192020 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:21.192020 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:21.193564 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:21.193564 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:21.193564 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 06 14:45:21.193564 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 06 14:45:21.193564 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 06 14:45:21.193564 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 06 14:45:21.193564 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 06 14:45:21.193564 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 06 14:45:21.193564 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 06 14:45:21.193564 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:21.197804 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 369/743] 10.222.0.23 () {56 vars in 1093 bytes} [Fri May 6 14:45:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:21.198694 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e69528-2197-4e3b-a757-11bcb3a95835 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:21.203636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 375/744] 10.222.0.23 () {56 vars in 1103 bytes} [Fri May 6 14:45:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:21.203636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9940b4-f622-4a5c-b2a9-5158bfecc22f None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:21.203636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9940b4-f622-4a5c-b2a9-5158bfecc22f None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:21.203636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9940b4-f622-4a5c-b2a9-5158bfecc22f None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:21.203636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 376/745] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:45:21.203636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9940b4-f622-4a5c-b2a9-5158bfecc22f None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:21.323191 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e69528-2197-4e3b-a757-11bcb3a95835 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:21.326089 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9940b4-f622-4a5c-b2a9-5158bfecc22f None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:21.367866 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaab480-7c84-4898-87af-1ad9df041b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:21.368696 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaab480-7c84-4898-87af-1ad9df041b63 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:21.368982 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaab480-7c84-4898-87af-1ad9df041b63 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:21.369252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaab480-7c84-4898-87af-1ad9df041b63 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:21.375473 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f2ef6-ab26-4368-85b1-f6b43c4b97fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:21.376016 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f2ef6-ab26-4368-85b1-f6b43c4b97fc None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:21.376372 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f2ef6-ab26-4368-85b1-f6b43c4b97fc None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:21.376757 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f2ef6-ab26-4368-85b1-f6b43c4b97fc None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:21.376893 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaab480-7c84-4898-87af-1ad9df041b63 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:21.377917 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733f2ef6-ab26-4368-85b1-f6b43c4b97fc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:21.378351 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaab480-7c84-4898-87af-1ad9df041b63 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:21.378508 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733f2ef6-ab26-4368-85b1-f6b43c4b97fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:21.381820 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733f2ef6-ab26-4368-85b1-f6b43c4b97fc None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:21.394458 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-fbaab480-7c84-4898-87af-1ad9df041b63 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:21.394458 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:21.394458 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:21.394458 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:21.394458 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:21.394458 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:21.394458 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:21.394458 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:21.394458 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:21.394458 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:21.394458 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:21.394458 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:21.394458 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 06 14:45:21.394458 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(project_id) May 06 14:45:21.394458 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 06 14:45:21.395395 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 06 14:45:21.395395 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:21.395395 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:21.395395 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:21.395395 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:21.395395 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:21.395395 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:21.395395 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:21.395395 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:21.395395 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:21.395395 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:21.395395 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:21.395395 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:21.395395 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:21.395395 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:21.395395 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:21.395395 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:21.396444 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 06 14:45:21.396444 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 06 14:45:21.396444 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 06 14:45:21.396444 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 06 14:45:21.396444 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 06 14:45:21.396444 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 06 14:45:21.396444 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:21.396444 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:21.396444 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 370/746] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:45:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:21.397118 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaab480-7c84-4898-87af-1ad9df041b63 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:21.412199 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 377/747] 10.222.0.23 () {56 vars in 1133 bytes} [Fri May 6 14:45:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:21.414709 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c524b5-f119-4e7b-b0ec-96fd37882175 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:21.415191 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c524b5-f119-4e7b-b0ec-96fd37882175 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:21.415656 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c524b5-f119-4e7b-b0ec-96fd37882175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:21.441816 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-c5c524b5-f119-4e7b-b0ec-96fd37882175 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:21.444475 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-c5c524b5-f119-4e7b-b0ec-96fd37882175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:21.513641 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaab480-7c84-4898-87af-1ad9df041b63 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:21.514343 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 378/748] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:21 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 06 14:45:21.515218 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c524b5-f119-4e7b-b0ec-96fd37882175 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:21.556575 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a43d56-7830-4453-9989-dcfe4f0fca5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:21.557554 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a43d56-7830-4453-9989-dcfe4f0fca5b None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:21.557941 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a43d56-7830-4453-9989-dcfe4f0fca5b None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:21.558348 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a43d56-7830-4453-9989-dcfe4f0fca5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:21.570005 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a43d56-7830-4453-9989-dcfe4f0fca5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:21.571593 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a43d56-7830-4453-9989-dcfe4f0fca5b None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:21.591057 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-f3a43d56-7830-4453-9989-dcfe4f0fca5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:21.591057 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:21.591057 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:21.591057 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:21.591057 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:21.591057 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:21.591057 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:21.591057 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:21.591057 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:21.591057 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:21.591057 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:21.591057 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:21.591057 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:21.591057 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:21.591057 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:21.592174 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:21.592174 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:21.592174 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:21.592174 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:21.592174 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:21.592174 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:21.592174 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:21.592174 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:21.592174 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:21.592174 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:21.592174 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:21.592174 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:21.592174 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:21.592174 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:21.592174 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:21.592174 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:21.592174 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:21.593353 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:21.593353 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:21.593353 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:21.593353 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:21.593832 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 371/749] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:21.594601 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a43d56-7830-4453-9989-dcfe4f0fca5b None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:21.632203 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c524b5-f119-4e7b-b0ec-96fd37882175 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:21.665613 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded6e01-1e44-46bb-ae5d-a35c049c6ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:21.666279 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded6e01-1e44-46bb-ae5d-a35c049c6ff6 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:21.666560 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded6e01-1e44-46bb-ae5d-a35c049c6ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:21.666831 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded6e01-1e44-46bb-ae5d-a35c049c6ff6 None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:21.667607 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dded6e01-1e44-46bb-ae5d-a35c049c6ff6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:21.667911 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dded6e01-1e44-46bb-ae5d-a35c049c6ff6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:21.669984 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dded6e01-1e44-46bb-ae5d-a35c049c6ff6 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:21.683092 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 379/750] 10.222.0.23 () {56 vars in 1148 bytes} [Fri May 6 14:45:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:21.688589 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded6e01-1e44-46bb-ae5d-a35c049c6ff6 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:21.691065 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a43d56-7830-4453-9989-dcfe4f0fca5b None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:21.724840 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d877fa7-ecc0-457b-bda0-da0387e50634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:21.725631 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d877fa7-ecc0-457b-bda0-da0387e50634 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:21.725926 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d877fa7-ecc0-457b-bda0-da0387e50634 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:21.726184 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d877fa7-ecc0-457b-bda0-da0387e50634 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:21.736179 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d877fa7-ecc0-457b-bda0-da0387e50634 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:21.737690 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d877fa7-ecc0-457b-bda0-da0387e50634 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:21.758346 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-8d877fa7-ecc0-457b-bda0-da0387e50634 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:21.758346 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:21.758346 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:21.758346 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:21.758346 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:21.758346 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:21.758346 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:21.758346 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:21.758346 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:21.758346 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:21.758346 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:21.758346 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:21.758346 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:21.758346 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:21.758346 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:21.759347 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:21.759347 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:21.759347 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:21.759347 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:21.759347 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:21.759347 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:21.759347 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:21.759347 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:21.759347 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:21.759347 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:21.759347 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:21.759347 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:21.759347 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:21.759347 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:21.759347 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:21.759347 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:21.759347 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:21.761443 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:21.761443 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:21.761443 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:21.761443 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:21.761443 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 372/751] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:21.761443 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d877fa7-ecc0-457b-bda0-da0387e50634 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:21.793951 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded6e01-1e44-46bb-ae5d-a35c049c6ff6 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:21.828576 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86f5c57-4cf5-4277-9c26-7267c5004980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:21.829274 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f5c57-4cf5-4277-9c26-7267c5004980 None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:21.829562 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f5c57-4cf5-4277-9c26-7267c5004980 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:21.829845 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f5c57-4cf5-4277-9c26-7267c5004980 None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:21.830568 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86f5c57-4cf5-4277-9c26-7267c5004980 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:21.831925 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86f5c57-4cf5-4277-9c26-7267c5004980 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:21.852976 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 380/752] 10.222.0.23 () {60 vars in 1141 bytes} [Fri May 6 14:45:21 2022] POST /identity/v3/services => generated 240 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:45:21.852976 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86f5c57-4cf5-4277-9c26-7267c5004980 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:21.870598 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d877fa7-ecc0-457b-bda0-da0387e50634 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:21.918662 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2a618b-8a7f-418d-88b1-98f4482fd891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:21.919896 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a618b-8a7f-418d-88b1-98f4482fd891 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:21.920200 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a618b-8a7f-418d-88b1-98f4482fd891 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:21.920479 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a618b-8a7f-418d-88b1-98f4482fd891 None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:21.921260 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2a618b-8a7f-418d-88b1-98f4482fd891 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:21.921660 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2a618b-8a7f-418d-88b1-98f4482fd891 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:21.923834 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2a618b-8a7f-418d-88b1-98f4482fd891 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:21.947079 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 373/753] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:21.948167 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2a618b-8a7f-418d-88b1-98f4482fd891 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:22.005991 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86f5c57-4cf5-4277-9c26-7267c5004980 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:22.050976 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b965b-f7c5-4fe8-84e6-8b29a30db9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:22.051644 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b965b-f7c5-4fe8-84e6-8b29a30db9f9 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:22.052527 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b965b-f7c5-4fe8-84e6-8b29a30db9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:22.052869 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b965b-f7c5-4fe8-84e6-8b29a30db9f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:22.053662 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678b965b-f7c5-4fe8-84e6-8b29a30db9f9 None admin] RBAC: Adding query filter params (role.id=6dcd35d1a83748759d6ef9f026077835, scope.project.id=d5398feee6eb490cb49bb261448a1a9b, user.id=bf9eed62c8f1409282c2af6488426a0f) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:22.053978 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678b965b-f7c5-4fe8-84e6-8b29a30db9f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:22.055413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678b965b-f7c5-4fe8-84e6-8b29a30db9f9 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:22.061207 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 381/754] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:45:21 2022] GET /identity/v3/role_assignments?user.id=bf9eed62c8f1409282c2af6488426a0f&scope.project.id=d5398feee6eb490cb49bb261448a1a9b&role.id=6dcd35d1a83748759d6ef9f026077835 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:22.062099 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b965b-f7c5-4fe8-84e6-8b29a30db9f9 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:22.097089 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2a618b-8a7f-418d-88b1-98f4482fd891 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:22.141037 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dfdc78-d19d-4cd7-b9c5-9f3cf497d16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:22.141738 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dfdc78-d19d-4cd7-b9c5-9f3cf497d16c None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:22.141966 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dfdc78-d19d-4cd7-b9c5-9f3cf497d16c None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:22.142174 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dfdc78-d19d-4cd7-b9c5-9f3cf497d16c None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:22.142934 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dfdc78-d19d-4cd7-b9c5-9f3cf497d16c None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:22.143245 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dfdc78-d19d-4cd7-b9c5-9f3cf497d16c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:22.144710 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dfdc78-d19d-4cd7-b9c5-9f3cf497d16c None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:22.168541 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 374/755] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:22.175274 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dfdc78-d19d-4cd7-b9c5-9f3cf497d16c None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:22.228069 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b965b-f7c5-4fe8-84e6-8b29a30db9f9 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:22.291686 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd7bbf9-8755-4d5c-bd75-14e7b84e34e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:22.292522 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7bbf9-8755-4d5c-bd75-14e7b84e34e1 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:22.292900 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7bbf9-8755-4d5c-bd75-14e7b84e34e1 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:22.293112 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7bbf9-8755-4d5c-bd75-14e7b84e34e1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:22.303686 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd7bbf9-8755-4d5c-bd75-14e7b84e34e1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:22.304901 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dfdc78-d19d-4cd7-b9c5-9f3cf497d16c None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:22.305171 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd7bbf9-8755-4d5c-bd75-14e7b84e34e1 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:22.321634 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-afd7bbf9-8755-4d5c-bd75-14e7b84e34e1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 06 14:45:22.321634 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:22.321634 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:22.321634 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:22.321634 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:22.321634 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:22.321634 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:22.321634 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:22.321634 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:22.321634 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:22.321634 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:22.321634 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:22.321634 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 06 14:45:22.321634 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_user(user_id) May 06 14:45:22.321634 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 06 14:45:22.321634 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 06 14:45:22.322618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:22.322618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:22.322618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 06 14:45:22.322618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:22.322618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 06 14:45:22.322618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:22.322618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:22.322618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:22.322618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:22.322618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:22.322618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:22.322618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:22.322618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:22.322618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:22.322618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:22.322618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:22.322618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:22.322618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:22.323539 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:22.323539 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:22.323539 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 06 14:45:22.323539 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 06 14:45:22.323539 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 06 14:45:22.323539 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 06 14:45:22.323539 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 06 14:45:22.323539 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 06 14:45:22.323539 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 06 14:45:22.323539 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:22.323539 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 382/756] 10.222.0.23 () {56 vars in 1117 bytes} [Fri May 6 14:45:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:22.328274 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd7bbf9-8755-4d5c-bd75-14e7b84e34e1 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:22.375767 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7c6814-2dd4-4c25-b215-437194b43c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:22.376561 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c6814-2dd4-4c25-b215-437194b43c40 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:22.377900 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c6814-2dd4-4c25-b215-437194b43c40 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:22.378924 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c6814-2dd4-4c25-b215-437194b43c40 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:22.393848 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7c6814-2dd4-4c25-b215-437194b43c40 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:22.395240 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7c6814-2dd4-4c25-b215-437194b43c40 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:22.425877 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-df7c6814-2dd4-4c25-b215-437194b43c40 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:22.425877 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:22.425877 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:22.425877 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:22.425877 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:22.425877 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:22.425877 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:22.425877 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:22.425877 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:22.425877 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:22.425877 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:22.425877 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:22.425877 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 06 14:45:22.425877 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(project_id) May 06 14:45:22.425877 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 06 14:45:22.426891 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 06 14:45:22.426891 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:22.426891 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:22.426891 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:22.426891 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:22.426891 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:22.426891 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:22.426891 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:22.426891 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:22.426891 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:22.426891 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:22.426891 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:22.426891 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:22.426891 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:22.426891 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:22.426891 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:22.426891 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:22.428111 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 06 14:45:22.428111 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 06 14:45:22.428111 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 06 14:45:22.428111 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 06 14:45:22.428111 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 06 14:45:22.428111 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 06 14:45:22.428111 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:22.428111 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:22.428111 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 375/757] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:45:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:22.434078 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7c6814-2dd4-4c25-b215-437194b43c40 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:22.495354 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd7bbf9-8755-4d5c-bd75-14e7b84e34e1 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:22.530184 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18d3984-f001-434c-b3cd-b4ee215f5e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:22.531118 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d3984-f001-434c-b3cd-b4ee215f5e2a None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:22.531396 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d3984-f001-434c-b3cd-b4ee215f5e2a None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:22.531667 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d3984-f001-434c-b3cd-b4ee215f5e2a None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:22.532499 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18d3984-f001-434c-b3cd-b4ee215f5e2a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:22.532834 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18d3984-f001-434c-b3cd-b4ee215f5e2a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:22.534260 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18d3984-f001-434c-b3cd-b4ee215f5e2a None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:22.555535 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 383/758] 10.222.0.23 () {56 vars in 1149 bytes} [Fri May 6 14:45:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:22.556974 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6dd4e0-33bb-4efa-8c52-16d5a60e2f3a None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:22.557185 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6dd4e0-33bb-4efa-8c52-16d5a60e2f3a None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:22.557458 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6dd4e0-33bb-4efa-8c52-16d5a60e2f3a None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:22.562963 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 384/759] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:45:22.564220 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7c6814-2dd4-4c25-b215-437194b43c40 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:22.564424 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6dd4e0-33bb-4efa-8c52-16d5a60e2f3a None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:22.600151 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855cbc31-ac6d-4fc1-aac0-128f078e04d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:22.601226 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855cbc31-ac6d-4fc1-aac0-128f078e04d0 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:22.601714 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855cbc31-ac6d-4fc1-aac0-128f078e04d0 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:22.602188 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855cbc31-ac6d-4fc1-aac0-128f078e04d0 None admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:22.603186 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855cbc31-ac6d-4fc1-aac0-128f078e04d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:22.603692 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855cbc31-ac6d-4fc1-aac0-128f078e04d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:22.605339 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855cbc31-ac6d-4fc1-aac0-128f078e04d0 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:22.619202 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 376/760] 10.222.0.23 () {56 vars in 1148 bytes} [Fri May 6 14:45:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:22.620823 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6797f-b665-4e32-8f5e-ce40d679bfce None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:22.621352 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6797f-b665-4e32-8f5e-ce40d679bfce None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:22.621825 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6797f-b665-4e32-8f5e-ce40d679bfce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:22.638639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-a1d6797f-b665-4e32-8f5e-ce40d679bfce None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:22.640114 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-a1d6797f-b665-4e32-8f5e-ce40d679bfce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:22.674206 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6dd4e0-33bb-4efa-8c52-16d5a60e2f3a None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:22.709294 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ff085-e841-479a-bf77-11c8d7db000c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:22.709954 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ff085-e841-479a-bf77-11c8d7db000c None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:22.710241 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ff085-e841-479a-bf77-11c8d7db000c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:22.710520 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ff085-e841-479a-bf77-11c8d7db000c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:22.712949 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 377/761] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:45:22.714010 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d6797f-b665-4e32-8f5e-ce40d679bfce None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:22.781623 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328ff085-e841-479a-bf77-11c8d7db000c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:22.783077 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328ff085-e841-479a-bf77-11c8d7db000c None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:22.814124 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d6797f-b665-4e32-8f5e-ce40d679bfce None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:22.848360 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467150ba-2f29-49d7-a2ee-23e3077625a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:22.849668 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467150ba-2f29-49d7-a2ee-23e3077625a6 None admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:22.850303 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467150ba-2f29-49d7-a2ee-23e3077625a6 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:22.850792 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467150ba-2f29-49d7-a2ee-23e3077625a6 None admin] PATH_INFO: `/v3/projects/d5398feee6eb490cb49bb261448a1a9b/users/fc8cf09642ec4e10a80badcf8b585ef8/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:22.853053 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467150ba-2f29-49d7-a2ee-23e3077625a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=d5398feee6eb490cb49bb261448a1a9b, user_id=fc8cf09642ec4e10a80badcf8b585ef8, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:22.854709 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467150ba-2f29-49d7-a2ee-23e3077625a6 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:22.869420 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 378/762] 10.222.0.23 () {58 vars in 1439 bytes} [Fri May 6 14:45:22 2022] PUT /identity/v3/projects/d5398feee6eb490cb49bb261448a1a9b/users/fc8cf09642ec4e10a80badcf8b585ef8/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:45:22.870650 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467150ba-2f29-49d7-a2ee-23e3077625a6 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:22.885507 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 385/763] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:45:22 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 06 14:45:22.890790 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ff085-e841-479a-bf77-11c8d7db000c None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:23.012544 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ff085-e841-479a-bf77-11c8d7db000c None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:23.049264 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd15a5f-8284-4868-a27e-b5bd12c8fa17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:23.050069 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd15a5f-8284-4868-a27e-b5bd12c8fa17 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:23.050353 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd15a5f-8284-4868-a27e-b5bd12c8fa17 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:23.050632 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd15a5f-8284-4868-a27e-b5bd12c8fa17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:23.061908 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd15a5f-8284-4868-a27e-b5bd12c8fa17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:23.063430 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd15a5f-8284-4868-a27e-b5bd12c8fa17 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:23.084618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-4dd15a5f-8284-4868-a27e-b5bd12c8fa17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:23.084618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:23.084618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:23.084618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:23.084618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:23.084618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:23.084618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:23.084618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:23.084618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:23.084618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:23.084618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:23.084618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:23.084618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:23.084618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:23.084618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:23.085636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:23.085636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:23.085636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:23.085636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:23.085636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:23.085636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:23.085636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:23.085636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:23.085636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:23.085636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:23.085636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:23.085636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:23.085636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:23.085636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:23.085636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:23.085636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:23.085636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:23.086529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:23.086529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:23.086529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:23.086529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:23.086529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 386/764] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:45:23.091523 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd15a5f-8284-4868-a27e-b5bd12c8fa17 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:23.135256 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467150ba-2f29-49d7-a2ee-23e3077625a6 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:23.173128 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bc743d-9ef0-4ea5-adc6-cf37b906e9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:23.173986 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc743d-9ef0-4ea5-adc6-cf37b906e9fc None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:23.174268 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc743d-9ef0-4ea5-adc6-cf37b906e9fc None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:23.174533 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc743d-9ef0-4ea5-adc6-cf37b906e9fc None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:23.175240 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bc743d-9ef0-4ea5-adc6-cf37b906e9fc None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:23.176529 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bc743d-9ef0-4ea5-adc6-cf37b906e9fc None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:23.189661 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-b1bc743d-9ef0-4ea5-adc6-cf37b906e9fc None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 06 14:45:23.189661 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:23.189661 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 06 14:45:23.189661 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 06 14:45:23.189661 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 06 14:45:23.189661 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 06 14:45:23.189661 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 06 14:45:23.189661 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:23.189661 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 06 14:45:23.189661 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:23.189661 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 06 14:45:23.189661 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:23.189661 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:23.189661 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:23.189661 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:23.189661 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:23.190630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:23.190630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:23.190630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:23.190630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:23.190630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:23.190630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:23.190630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:23.190630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 06 14:45:23.190630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(service_id) May 06 14:45:23.190630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 06 14:45:23.190630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 06 14:45:23.190630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:23.190630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:23.190630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:23.190630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:23.190630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:23.191474 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:23.191474 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:23.191474 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:23.191474 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:23.191474 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:23.191474 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:23.191474 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:23.191474 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:23.191474 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:23.191474 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:23.191474 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:23.191474 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 06 14:45:23.191474 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:23.191474 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 06 14:45:23.191474 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:23.191474 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 379/765] 10.222.0.23 () {56 vars in 1129 bytes} [Fri May 6 14:45:22 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:23.197471 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bc743d-9ef0-4ea5-adc6-cf37b906e9fc None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:23.209907 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd15a5f-8284-4868-a27e-b5bd12c8fa17 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:23.247450 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e5448e-f3de-4688-990f-54802b01b202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:23.248114 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5448e-f3de-4688-990f-54802b01b202 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:23.248391 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5448e-f3de-4688-990f-54802b01b202 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:23.248688 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5448e-f3de-4688-990f-54802b01b202 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:23.249405 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e5448e-f3de-4688-990f-54802b01b202 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:23.249703 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e5448e-f3de-4688-990f-54802b01b202 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:23.251054 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e5448e-f3de-4688-990f-54802b01b202 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:23.263353 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 387/766] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:23.268406 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e5448e-f3de-4688-990f-54802b01b202 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:23.304125 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bc743d-9ef0-4ea5-adc6-cf37b906e9fc None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:23.339152 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8ef9a5-632e-4ca5-8584-a300d37313fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:23.339926 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ef9a5-632e-4ca5-8584-a300d37313fa None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:23.340311 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ef9a5-632e-4ca5-8584-a300d37313fa None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:23.340653 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ef9a5-632e-4ca5-8584-a300d37313fa None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:23.341475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8ef9a5-632e-4ca5-8584-a300d37313fa None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:23.341830 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8ef9a5-632e-4ca5-8584-a300d37313fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:23.343456 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8ef9a5-632e-4ca5-8584-a300d37313fa None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:23.349673 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 380/767] 10.222.0.23 () {56 vars in 1124 bytes} [Fri May 6 14:45:23 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:23.355075 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8ef9a5-632e-4ca5-8584-a300d37313fa None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:23.371094 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e5448e-f3de-4688-990f-54802b01b202 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:23.405253 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b7bd0a-a220-4765-bc53-4f169e0b89a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:23.406331 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b7bd0a-a220-4765-bc53-4f169e0b89a4 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:23.406610 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b7bd0a-a220-4765-bc53-4f169e0b89a4 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:23.406881 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b7bd0a-a220-4765-bc53-4f169e0b89a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:23.415178 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b7bd0a-a220-4765-bc53-4f169e0b89a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:23.416652 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b7bd0a-a220-4765-bc53-4f169e0b89a4 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:23.432053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-72b7bd0a-a220-4765-bc53-4f169e0b89a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:23.432053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:23.432053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:23.432053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:23.432053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:23.432053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:23.432053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:23.432053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:23.432053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:23.432053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:23.432053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:23.432053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:23.432053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 06 14:45:23.432053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(project_id) May 06 14:45:23.432053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 06 14:45:23.434247 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 06 14:45:23.434247 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:23.434247 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:23.434247 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:23.434247 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:23.434247 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:23.434247 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:23.434247 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:23.434247 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:23.434247 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:23.434247 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:23.434247 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:23.434247 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:23.434247 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:23.434247 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:23.434247 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:23.434247 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:23.436349 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 06 14:45:23.436349 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 06 14:45:23.436349 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 06 14:45:23.436349 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 06 14:45:23.436349 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 06 14:45:23.436349 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 06 14:45:23.436349 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:23.436349 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:23.436349 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 388/768] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:45:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:45:23.440638 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b7bd0a-a220-4765-bc53-4f169e0b89a4 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:23.459237 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8ef9a5-632e-4ca5-8584-a300d37313fa None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:23.496808 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357b1b9e-ebd8-4ffd-b52e-70d7c2db25b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:23.498093 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b1b9e-ebd8-4ffd-b52e-70d7c2db25b3 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:23.498386 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b1b9e-ebd8-4ffd-b52e-70d7c2db25b3 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:23.498650 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b1b9e-ebd8-4ffd-b52e-70d7c2db25b3 None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:23.499357 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357b1b9e-ebd8-4ffd-b52e-70d7c2db25b3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:23.499664 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357b1b9e-ebd8-4ffd-b52e-70d7c2db25b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:23.501068 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357b1b9e-ebd8-4ffd-b52e-70d7c2db25b3 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:23.510364 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 381/769] 10.222.0.23 () {56 vars in 1124 bytes} [Fri May 6 14:45:23 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:23.519447 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357b1b9e-ebd8-4ffd-b52e-70d7c2db25b3 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:23.585211 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b7bd0a-a220-4765-bc53-4f169e0b89a4 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:23.654620 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc0763-77f0-46b4-858c-caf6ed6eaa78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:23.655273 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc0763-77f0-46b4-858c-caf6ed6eaa78 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:23.655809 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc0763-77f0-46b4-858c-caf6ed6eaa78 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:23.656081 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc0763-77f0-46b4-858c-caf6ed6eaa78 None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:23.656941 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dc0763-77f0-46b4-858c-caf6ed6eaa78 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:23.657246 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dc0763-77f0-46b4-858c-caf6ed6eaa78 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:23.658713 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dc0763-77f0-46b4-858c-caf6ed6eaa78 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:23.682294 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 389/770] 10.222.0.23 () {56 vars in 1148 bytes} [Fri May 6 14:45:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:23.688201 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc0763-77f0-46b4-858c-caf6ed6eaa78 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:23.709149 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357b1b9e-ebd8-4ffd-b52e-70d7c2db25b3 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:23.751068 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c16b4-8562-4e30-b683-3352f868e50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:23.752096 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c16b4-8562-4e30-b683-3352f868e50c None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:23.752573 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c16b4-8562-4e30-b683-3352f868e50c None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:23.753078 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c16b4-8562-4e30-b683-3352f868e50c None admin] PATH_INFO: `/v3/endpoints` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:23.754054 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680c16b4-8562-4e30-b683-3352f868e50c None admin] RBAC: Adding query filter params (interface=public, service_id=331bfcf1ed504763bebb23066c0ab273, region_id=RegionOne) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:23.754595 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680c16b4-8562-4e30-b683-3352f868e50c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:23.757036 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680c16b4-8562-4e30-b683-3352f868e50c None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:23.765122 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 382/771] 10.222.0.23 () {56 vars in 1251 bytes} [Fri May 6 14:45:23 2022] GET /identity/v3/endpoints?service_id=331bfcf1ed504763bebb23066c0ab273&interface=public®ion_id=RegionOne => generated 194 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:23.770378 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c16b4-8562-4e30-b683-3352f868e50c None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:23.870344 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc0763-77f0-46b4-858c-caf6ed6eaa78 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:23.932290 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c947e7d-5c78-43d4-a447-15ce888bef2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:23.933773 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c947e7d-5c78-43d4-a447-15ce888bef2d None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:23.934272 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c947e7d-5c78-43d4-a447-15ce888bef2d None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:23.934745 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c947e7d-5c78-43d4-a447-15ce888bef2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:23.935805 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c947e7d-5c78-43d4-a447-15ce888bef2d None admin] RBAC: Adding query filter params (role.id=f5641817ee56438496afc6135d37a4c0, scope.project.id=d5398feee6eb490cb49bb261448a1a9b, user.id=a7dc1bc53c59423e934f7831ea022c73) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:23.936313 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c947e7d-5c78-43d4-a447-15ce888bef2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:23.939132 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c947e7d-5c78-43d4-a447-15ce888bef2d None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:23.950323 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c16b4-8562-4e30-b683-3352f868e50c None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:23.952431 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 390/772] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:45:23 2022] GET /identity/v3/role_assignments?user.id=a7dc1bc53c59423e934f7831ea022c73&scope.project.id=d5398feee6eb490cb49bb261448a1a9b&role.id=f5641817ee56438496afc6135d37a4c0 => generated 259 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:23.987084 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7f6f10-fd54-4ce7-9efe-6bc0532db43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:23.988149 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7f6f10-fd54-4ce7-9efe-6bc0532db43c None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:23.988747 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7f6f10-fd54-4ce7-9efe-6bc0532db43c None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:23.989779 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7f6f10-fd54-4ce7-9efe-6bc0532db43c None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:23.991078 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7f6f10-fd54-4ce7-9efe-6bc0532db43c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:23.992711 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7f6f10-fd54-4ce7-9efe-6bc0532db43c None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:23.999140 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 383/773] 10.222.0.23 () {56 vars in 1087 bytes} [Fri May 6 14:45:23 2022] GET /identity/v3/services => generated 772 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:24.625007 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95911c-ba06-44c6-b1b0-3ab6b44b38d2 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:24.625398 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95911c-ba06-44c6-b1b0-3ab6b44b38d2 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:24.625595 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95911c-ba06-44c6-b1b0-3ab6b44b38d2 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:24.627059 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 391/774] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:45:24.635177 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63149035-86b3-46d4-a9dd-4c8fe669c789 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:24.635177 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63149035-86b3-46d4-a9dd-4c8fe669c789 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:24.635177 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63149035-86b3-46d4-a9dd-4c8fe669c789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:24.646786 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667216e2-be89-48a5-a600-2c3b24f8675c None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:24.647272 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667216e2-be89-48a5-a600-2c3b24f8675c None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:24.647560 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667216e2-be89-48a5-a600-2c3b24f8675c None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:24.649193 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 392/775] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:45:24.654078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39407e3-df76-47df-98ac-6b6ff1e47020 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:24.654370 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39407e3-df76-47df-98ac-6b6ff1e47020 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:24.654650 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39407e3-df76-47df-98ac-6b6ff1e47020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:24.656740 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-63149035-86b3-46d4-a9dd-4c8fe669c789 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:24.658129 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-63149035-86b3-46d4-a9dd-4c8fe669c789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:24.685612 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-d39407e3-df76-47df-98ac-6b6ff1e47020 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:24.686792 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-d39407e3-df76-47df-98ac-6b6ff1e47020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:24.727203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 384/776] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:45:24.752931 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63149035-86b3-46d4-a9dd-4c8fe669c789 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:24.810464 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 393/777] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:45:24.852331 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63149035-86b3-46d4-a9dd-4c8fe669c789 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:24.858056 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39407e3-df76-47df-98ac-6b6ff1e47020 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:24.887900 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00392049-8fe8-4d21-a98a-3494f4c3ddad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:24.889057 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00392049-8fe8-4d21-a98a-3494f4c3ddad None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:24.889557 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00392049-8fe8-4d21-a98a-3494f4c3ddad None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:24.890026 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00392049-8fe8-4d21-a98a-3494f4c3ddad None admin] PATH_INFO: `/v3/services/network` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:24.890863 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00392049-8fe8-4d21-a98a-3494f4c3ddad None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:24.892433 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00392049-8fe8-4d21-a98a-3494f4c3ddad None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:24.904553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-00392049-8fe8-4d21-a98a-3494f4c3ddad None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 06 14:45:24.904553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:24.904553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 06 14:45:24.904553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 06 14:45:24.904553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 06 14:45:24.904553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 06 14:45:24.904553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 06 14:45:24.904553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:24.904553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 06 14:45:24.904553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:24.904553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 06 14:45:24.904553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:24.904553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:24.904553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:24.904553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:24.904553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:24.904553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:24.905915 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:24.905915 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:24.905915 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:24.905915 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:24.905915 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:24.905915 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:24.905915 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 06 14:45:24.905915 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(service_id) May 06 14:45:24.905915 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 06 14:45:24.905915 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 06 14:45:24.905915 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:24.905915 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:24.905915 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:24.905915 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:24.905915 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:24.905915 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:24.906759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:24.906759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:24.906759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:24.906759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:24.906759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:24.906759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:24.906759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:24.906759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:24.906759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:24.906759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:24.906759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 06 14:45:24.906759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:24.906759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 06 14:45:24.906759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:24.909622 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 385/778] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:45:24 2022] GET /identity/v3/services/network => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:45:24.915585 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00392049-8fe8-4d21-a98a-3494f4c3ddad None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:25.018765 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00392049-8fe8-4d21-a98a-3494f4c3ddad None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:25.052834 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39407e3-df76-47df-98ac-6b6ff1e47020 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:25.054493 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412fafe5-7920-44db-b8a0-55c8af08bd6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:25.055405 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412fafe5-7920-44db-b8a0-55c8af08bd6c None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:25.055881 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412fafe5-7920-44db-b8a0-55c8af08bd6c None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:25.056349 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412fafe5-7920-44db-b8a0-55c8af08bd6c None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:25.057277 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412fafe5-7920-44db-b8a0-55c8af08bd6c None admin] RBAC: Adding query filter params (name=network) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:25.057792 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412fafe5-7920-44db-b8a0-55c8af08bd6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:25.059346 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412fafe5-7920-44db-b8a0-55c8af08bd6c None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:25.064418 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 386/779] 10.222.0.23 () {56 vars in 1112 bytes} [Fri May 6 14:45:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:25.070975 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412fafe5-7920-44db-b8a0-55c8af08bd6c None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:25.184562 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412fafe5-7920-44db-b8a0-55c8af08bd6c None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:25.221728 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56acdcad-d99c-49d7-aecf-b9321965d612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:25.222819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56acdcad-d99c-49d7-aecf-b9321965d612 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:25.223298 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56acdcad-d99c-49d7-aecf-b9321965d612 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:25.223846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56acdcad-d99c-49d7-aecf-b9321965d612 None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:25.224775 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56acdcad-d99c-49d7-aecf-b9321965d612 None admin] RBAC: Adding query filter params (type=network) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:25.225292 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56acdcad-d99c-49d7-aecf-b9321965d612 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:25.226878 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56acdcad-d99c-49d7-aecf-b9321965d612 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:25.232427 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 387/780] 10.222.0.23 () {56 vars in 1112 bytes} [Fri May 6 14:45:25 2022] GET /identity/v3/services?type=network => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:25.238853 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56acdcad-d99c-49d7-aecf-b9321965d612 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:25.245406 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb12ae0-56d9-41f7-8d1b-e41df8bf4941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:25.246204 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb12ae0-56d9-41f7-8d1b-e41df8bf4941 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:25.246481 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb12ae0-56d9-41f7-8d1b-e41df8bf4941 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:25.246751 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb12ae0-56d9-41f7-8d1b-e41df8bf4941 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:25.258322 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb12ae0-56d9-41f7-8d1b-e41df8bf4941 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:45:25.265918 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-4cb12ae0-56d9-41f7-8d1b-e41df8bf4941 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:45:25.265918 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:25.265918 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:25.265918 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:25.265918 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:25.265918 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:25.265918 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:25.265918 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:25.265918 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:25.265918 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:25.265918 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:25.265918 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:25.265918 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:45:25.265918 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:45:25.265918 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:45:25.265918 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:45:25.266870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:45:25.266870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:45:25.266870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:45:25.266870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:45:25.266870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:25.266870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:25.266870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:25.266870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:25.266870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:25.266870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:25.266870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:25.266870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:25.266870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:25.266870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:25.266870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:25.266870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:25.267698 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:25.267698 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:25.267698 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:25.267698 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:25.267698 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:45:25.267698 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:45:25.267698 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:45:25.267698 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:45:25.267698 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:45:25.267698 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:45:25.267698 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:45:25.267698 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:25.267698 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 394/781] 10.222.0.23 () {56 vars in 1096 bytes} [Fri May 6 14:45:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:25.277844 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb12ae0-56d9-41f7-8d1b-e41df8bf4941 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:25.358002 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56acdcad-d99c-49d7-aecf-b9321965d612 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:25.396272 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2599807-afe5-4754-a58c-0908f6907b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:25.397434 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2599807-afe5-4754-a58c-0908f6907b21 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:25.398061 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2599807-afe5-4754-a58c-0908f6907b21 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:25.398539 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2599807-afe5-4754-a58c-0908f6907b21 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:25.399744 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2599807-afe5-4754-a58c-0908f6907b21 None admin] RBAC: Adding query filter params (interface=public, service_id=9af3d66e1421479285be9c176632f6ee, region_id=RegionOne) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:25.400270 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2599807-afe5-4754-a58c-0908f6907b21 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:25.401938 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2599807-afe5-4754-a58c-0908f6907b21 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:25.409426 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 388/782] 10.222.0.23 () {56 vars in 1251 bytes} [Fri May 6 14:45:25 2022] GET /identity/v3/endpoints?service_id=9af3d66e1421479285be9c176632f6ee&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:25.416361 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2599807-afe5-4754-a58c-0908f6907b21 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:25.486753 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb12ae0-56d9-41f7-8d1b-e41df8bf4941 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:25.529160 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2599807-afe5-4754-a58c-0908f6907b21 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:25.569831 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f68dc2e-a6e8-4ccd-8792-15d56120b4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:25.570852 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f68dc2e-a6e8-4ccd-8792-15d56120b4a6 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:25.571331 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f68dc2e-a6e8-4ccd-8792-15d56120b4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:25.571947 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f68dc2e-a6e8-4ccd-8792-15d56120b4a6 None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:25.573001 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f68dc2e-a6e8-4ccd-8792-15d56120b4a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:25.574727 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f68dc2e-a6e8-4ccd-8792-15d56120b4a6 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:25.581646 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 389/783] 10.222.0.23 () {56 vars in 1087 bytes} [Fri May 6 14:45:25 2022] GET /identity/v3/services => generated 772 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:25.593507 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b05297b-6d60-4a05-8f54-e54232bd3c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:25.594211 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b05297b-6d60-4a05-8f54-e54232bd3c5a None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:25.594493 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b05297b-6d60-4a05-8f54-e54232bd3c5a None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:25.594766 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b05297b-6d60-4a05-8f54-e54232bd3c5a None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:25.595531 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b05297b-6d60-4a05-8f54-e54232bd3c5a None admin] RBAC: Adding query filter params (name=admin) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:25.595834 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b05297b-6d60-4a05-8f54-e54232bd3c5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:25.597270 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b05297b-6d60-4a05-8f54-e54232bd3c5a None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:25.612642 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 395/784] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:45:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:25.618694 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f68dc2e-a6e8-4ccd-8792-15d56120b4a6 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:25.734867 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f68dc2e-a6e8-4ccd-8792-15d56120b4a6 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:25.775886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5386d-07a3-4253-acf5-6bab889973a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:25.777099 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5386d-07a3-4253-acf5-6bab889973a9 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:25.777590 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5386d-07a3-4253-acf5-6bab889973a9 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:25.778096 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5386d-07a3-4253-acf5-6bab889973a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:25.790384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db5386d-07a3-4253-acf5-6bab889973a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:25.792165 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db5386d-07a3-4253-acf5-6bab889973a9 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:25.810806 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-4db5386d-07a3-4253-acf5-6bab889973a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:25.810806 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:25.810806 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:25.810806 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:25.810806 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:25.810806 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:25.810806 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:25.810806 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:25.810806 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:25.810806 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:25.810806 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:25.810806 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:25.810806 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:25.810806 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:25.810806 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:25.811767 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:25.811767 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:25.811767 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:25.811767 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:25.811767 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:25.811767 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:25.811767 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:25.811767 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:25.811767 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:25.811767 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:25.811767 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:25.811767 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:25.811767 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:25.811767 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:25.811767 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:25.811767 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:25.811767 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:25.812704 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:25.812704 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:25.812704 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:25.812704 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:25.815040 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 390/785] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:25.821248 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b05297b-6d60-4a05-8f54-e54232bd3c5a None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:25.942116 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f2dd12-1541-41ce-bfd2-3c258fc93c69 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:25.942555 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f2dd12-1541-41ce-bfd2-3c258fc93c69 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:25.942881 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f2dd12-1541-41ce-bfd2-3c258fc93c69 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:25.944954 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 391/786] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:45:25.950641 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b655ac9-3304-41f2-93d1-f39ea5808cb1 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:25.951200 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b655ac9-3304-41f2-93d1-f39ea5808cb1 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:25.951674 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b655ac9-3304-41f2-93d1-f39ea5808cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:25.968978 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b05297b-6d60-4a05-8f54-e54232bd3c5a None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:25.970337 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-4b655ac9-3304-41f2-93d1-f39ea5808cb1 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:25.971707 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-4b655ac9-3304-41f2-93d1-f39ea5808cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:26.008503 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220e7aa9-9aff-4e06-b2a1-6bcbc6d77bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:26.009158 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e7aa9-9aff-4e06-b2a1-6bcbc6d77bd9 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:26.009440 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e7aa9-9aff-4e06-b2a1-6bcbc6d77bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:26.009716 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e7aa9-9aff-4e06-b2a1-6bcbc6d77bd9 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:26.010976 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220e7aa9-9aff-4e06-b2a1-6bcbc6d77bd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:26.010976 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220e7aa9-9aff-4e06-b2a1-6bcbc6d77bd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:26.012090 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220e7aa9-9aff-4e06-b2a1-6bcbc6d77bd9 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:26.028486 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 396/787] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:26.030022 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220e7aa9-9aff-4e06-b2a1-6bcbc6d77bd9 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:26.044626 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 392/788] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 06 14:45:26.069843 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b655ac9-3304-41f2-93d1-f39ea5808cb1 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:26.153306 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220e7aa9-9aff-4e06-b2a1-6bcbc6d77bd9 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:26.183489 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b655ac9-3304-41f2-93d1-f39ea5808cb1 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:26.199199 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5099bd39-b54a-4d22-b998-94ec909f103e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:26.200348 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099bd39-b54a-4d22-b998-94ec909f103e None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:26.200852 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099bd39-b54a-4d22-b998-94ec909f103e None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:26.201348 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099bd39-b54a-4d22-b998-94ec909f103e None admin] PATH_INFO: `/v3/users/swift` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:26.212337 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5099bd39-b54a-4d22-b998-94ec909f103e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:26.220627 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5099bd39-b54a-4d22-b998-94ec909f103e None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:26.220759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442e3e95-41cc-48b0-9ca1-21d7d2c822a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:26.220759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e3e95-41cc-48b0-9ca1-21d7d2c822a7 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:26.220759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e3e95-41cc-48b0-9ca1-21d7d2c822a7 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:26.220759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e3e95-41cc-48b0-9ca1-21d7d2c822a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:26.231620 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442e3e95-41cc-48b0-9ca1-21d7d2c822a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:45:26.237851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-5099bd39-b54a-4d22-b998-94ec909f103e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 06 14:45:26.237851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:26.237851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:26.237851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:26.237851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:26.237851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:26.237851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:26.237851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:26.237851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:26.237851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:26.237851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:26.237851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:26.237851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 06 14:45:26.237851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_user(user_id) May 06 14:45:26.237851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 06 14:45:26.237851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 06 14:45:26.238798 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:26.238798 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:26.238798 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 06 14:45:26.238798 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:26.238798 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 06 14:45:26.238798 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:26.238798 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:26.238798 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:26.238798 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:26.238798 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:26.238798 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:26.238798 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:26.238798 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:26.238798 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:26.238798 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:26.238798 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:26.238798 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:26.238798 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:26.240309 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:26.240309 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:26.240309 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 06 14:45:26.240309 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 06 14:45:26.240309 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 06 14:45:26.240309 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 06 14:45:26.240309 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 06 14:45:26.240309 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 06 14:45:26.240309 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 06 14:45:26.240309 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:26.240876 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-442e3e95-41cc-48b0-9ca1-21d7d2c822a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:45:26.240876 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:26.240876 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:26.240876 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:26.240876 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:26.240876 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:26.240876 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:26.240876 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:26.240876 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:26.240876 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:26.240876 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:26.240876 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:26.240876 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:45:26.240876 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:45:26.240876 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:45:26.240876 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:45:26.241715 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:45:26.241715 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:45:26.241715 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:45:26.241715 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:45:26.241715 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:26.241715 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:26.241715 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:26.241715 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:26.241715 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:26.241715 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:26.241715 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:26.241715 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:26.241715 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:26.241715 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:26.241715 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:26.241715 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:26.242573 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:26.242573 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:26.242573 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:26.242573 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:26.242573 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:45:26.242573 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:45:26.242573 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:45:26.242573 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:45:26.242573 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:45:26.242573 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:45:26.242573 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:45:26.242573 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:26.245350 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 393/789] 10.222.0.23 () {56 vars in 1096 bytes} [Fri May 6 14:45:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:26.246219 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 397/790] 10.222.0.23 () {56 vars in 1096 bytes} [Fri May 6 14:45:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:26.251629 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442e3e95-41cc-48b0-9ca1-21d7d2c822a7 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:26.266290 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5099bd39-b54a-4d22-b998-94ec909f103e None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:26.439537 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5099bd39-b54a-4d22-b998-94ec909f103e None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:26.451719 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442e3e95-41cc-48b0-9ca1-21d7d2c822a7 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:26.501856 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98246ad-0423-4fb7-b700-a3f4ddd6e0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:26.502533 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98246ad-0423-4fb7-b700-a3f4ddd6e0d3 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:26.502814 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98246ad-0423-4fb7-b700-a3f4ddd6e0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:26.503088 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98246ad-0423-4fb7-b700-a3f4ddd6e0d3 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:26.503825 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98246ad-0423-4fb7-b700-a3f4ddd6e0d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:26.504128 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98246ad-0423-4fb7-b700-a3f4ddd6e0d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:26.505819 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98246ad-0423-4fb7-b700-a3f4ddd6e0d3 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:26.511034 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a2ffd0-75ae-401e-853a-fe8bde69ef68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:26.511698 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2ffd0-75ae-401e-853a-fe8bde69ef68 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:26.511975 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2ffd0-75ae-401e-853a-fe8bde69ef68 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:26.512244 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2ffd0-75ae-401e-853a-fe8bde69ef68 None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:26.513096 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a2ffd0-75ae-401e-853a-fe8bde69ef68 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:26.513400 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a2ffd0-75ae-401e-853a-fe8bde69ef68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:26.514811 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a2ffd0-75ae-401e-853a-fe8bde69ef68 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:26.531360 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 398/791] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:45:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:26.541711 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98246ad-0423-4fb7-b700-a3f4ddd6e0d3 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:26.547615 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 394/792] 10.222.0.23 () {56 vars in 1135 bytes} [Fri May 6 14:45:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:26.553158 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a2ffd0-75ae-401e-853a-fe8bde69ef68 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:26.668634 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a2ffd0-75ae-401e-853a-fe8bde69ef68 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:26.713174 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b515e24-fb5d-41fe-814e-0a1d7953efdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:26.714000 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b515e24-fb5d-41fe-814e-0a1d7953efdc None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:26.714279 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b515e24-fb5d-41fe-814e-0a1d7953efdc None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:26.714552 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b515e24-fb5d-41fe-814e-0a1d7953efdc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:26.742454 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b515e24-fb5d-41fe-814e-0a1d7953efdc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:26.745432 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b515e24-fb5d-41fe-814e-0a1d7953efdc None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:26.768633 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98246ad-0423-4fb7-b700-a3f4ddd6e0d3 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:26.792716 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-9b515e24-fb5d-41fe-814e-0a1d7953efdc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:26.792716 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:26.792716 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:26.792716 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:26.792716 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:26.792716 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:26.792716 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:26.792716 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:26.792716 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:26.792716 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:26.792716 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:26.792716 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:26.792716 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:26.792716 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:26.792716 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:26.793633 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:26.793633 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:26.793633 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:26.793633 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:26.793633 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:26.793633 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:26.793633 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:26.793633 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:26.793633 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:26.793633 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:26.793633 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:26.793633 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:26.793633 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:26.793633 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:26.793633 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:26.793633 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:26.793633 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:26.794520 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:26.794520 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:26.794520 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:26.794520 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:26.796829 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 395/793] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:26.804493 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b515e24-fb5d-41fe-814e-0a1d7953efdc None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:26.811595 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb811b-be5a-472f-9692-ea2732139b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:26.812735 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb811b-be5a-472f-9692-ea2732139b6f None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:26.813221 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb811b-be5a-472f-9692-ea2732139b6f None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:26.813745 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb811b-be5a-472f-9692-ea2732139b6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:26.838674 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bb811b-be5a-472f-9692-ea2732139b6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:26.840466 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bb811b-be5a-472f-9692-ea2732139b6f None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:26.876624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-f4bb811b-be5a-472f-9692-ea2732139b6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:26.876624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:26.876624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:26.876624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:26.876624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:26.876624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:26.876624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:26.876624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:26.876624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:26.876624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:26.876624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:26.876624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:26.876624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:26.876624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:26.876624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:26.877562 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:26.877562 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:26.877562 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:26.877562 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:26.877562 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:26.877562 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:26.877562 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:26.877562 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:26.877562 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:26.877562 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:26.877562 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:26.877562 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:26.877562 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:26.877562 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:26.877562 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:26.877562 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:26.877562 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:26.878445 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:26.878445 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:26.878445 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:26.878445 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:26.882256 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 399/794] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:26.898033 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb811b-be5a-472f-9692-ea2732139b6f None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:26.918900 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b515e24-fb5d-41fe-814e-0a1d7953efdc None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:26.959068 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec0752-261b-45a4-9ecc-8ffbe3642a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:26.959734 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec0752-261b-45a4-9ecc-8ffbe3642a1a None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:26.960017 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec0752-261b-45a4-9ecc-8ffbe3642a1a None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:26.960295 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec0752-261b-45a4-9ecc-8ffbe3642a1a None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:26.961328 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ec0752-261b-45a4-9ecc-8ffbe3642a1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:26.961641 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ec0752-261b-45a4-9ecc-8ffbe3642a1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:26.963010 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ec0752-261b-45a4-9ecc-8ffbe3642a1a None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:26.975146 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 396/795] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:26.976433 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c40b5c-1265-4408-b8c9-dfd72fda8281 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:26.977062 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c40b5c-1265-4408-b8c9-dfd72fda8281 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:26.977364 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c40b5c-1265-4408-b8c9-dfd72fda8281 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:26.982547 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 397/796] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:45:26.982547 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c305a0-4ebc-42ca-a1b2-051cfd4aee0e None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:26.982547 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c305a0-4ebc-42ca-a1b2-051cfd4aee0e None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:26.982547 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c305a0-4ebc-42ca-a1b2-051cfd4aee0e None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:26.982547 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 398/797] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:45:26.982547 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c305a0-4ebc-42ca-a1b2-051cfd4aee0e None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:27.020275 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb811b-be5a-472f-9692-ea2732139b6f None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:27.054413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed00ae3-9aeb-47a9-887a-d383d8113c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:27.054986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed00ae3-9aeb-47a9-887a-d383d8113c73 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:27.055267 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed00ae3-9aeb-47a9-887a-d383d8113c73 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:27.055537 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed00ae3-9aeb-47a9-887a-d383d8113c73 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:27.056239 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed00ae3-9aeb-47a9-887a-d383d8113c73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:27.056547 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed00ae3-9aeb-47a9-887a-d383d8113c73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:27.057922 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed00ae3-9aeb-47a9-887a-d383d8113c73 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:27.073771 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 400/798] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:27.073771 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f101b8-c819-4781-8360-7f9d915912d4 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:27.073771 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f101b8-c819-4781-8360-7f9d915912d4 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:27.073771 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f101b8-c819-4781-8360-7f9d915912d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:27.086688 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c305a0-4ebc-42ca-a1b2-051cfd4aee0e None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:27.086973 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-63f101b8-c819-4781-8360-7f9d915912d4 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:27.088142 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-63f101b8-c819-4781-8360-7f9d915912d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:27.120790 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d3d08-bbc9-450a-afe5-40e445f701f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:27.121583 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d3d08-bbc9-450a-afe5-40e445f701f9 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:27.121862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d3d08-bbc9-450a-afe5-40e445f701f9 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:27.122136 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d3d08-bbc9-450a-afe5-40e445f701f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:27.130220 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29d3d08-bbc9-450a-afe5-40e445f701f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:27.131701 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29d3d08-bbc9-450a-afe5-40e445f701f9 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:27.150099 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-e29d3d08-bbc9-450a-afe5-40e445f701f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:27.150099 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:27.150099 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:27.150099 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:27.150099 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:27.150099 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:27.150099 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:27.150099 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:27.150099 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:27.150099 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:27.150099 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:27.150099 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:27.150099 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 06 14:45:27.150099 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(project_id) May 06 14:45:27.150099 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 06 14:45:27.151074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 06 14:45:27.151074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:27.151074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:27.151074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:27.151074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:27.151074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:27.151074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:27.151074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:27.151074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:27.151074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:27.151074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:27.151074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:27.151074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:27.151074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:27.151074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:27.151074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:27.151074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:27.152006 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 06 14:45:27.152006 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 06 14:45:27.152006 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 06 14:45:27.152006 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 06 14:45:27.152006 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 06 14:45:27.152006 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 06 14:45:27.152006 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:27.152006 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:27.157183 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 399/799] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:45:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:27.158865 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad9811-9315-42f2-bddf-569e9fca941b None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:27.159161 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad9811-9315-42f2-bddf-569e9fca941b None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:27.159439 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad9811-9315-42f2-bddf-569e9fca941b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:27.171798 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 401/800] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:45:27.177002 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-15ad9811-9315-42f2-bddf-569e9fca941b None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:27.178091 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-15ad9811-9315-42f2-bddf-569e9fca941b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:27.184576 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f101b8-c819-4781-8360-7f9d915912d4 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:27.248246 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 400/801] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:45:27.249118 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad9811-9315-42f2-bddf-569e9fca941b None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:27.301281 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f101b8-c819-4781-8360-7f9d915912d4 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:27.345683 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad9811-9315-42f2-bddf-569e9fca941b None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:27.351522 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de42a7b3-5bc3-4c68-818f-18665542b39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:27.352315 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42a7b3-5bc3-4c68-818f-18665542b39d None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:27.352621 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42a7b3-5bc3-4c68-818f-18665542b39d None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:27.352909 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42a7b3-5bc3-4c68-818f-18665542b39d None admin] PATH_INFO: `/v3/users/nova` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:27.360239 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de42a7b3-5bc3-4c68-818f-18665542b39d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:27.361708 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de42a7b3-5bc3-4c68-818f-18665542b39d None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:27.375857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-de42a7b3-5bc3-4c68-818f-18665542b39d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 06 14:45:27.375857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:27.375857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:27.375857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:27.375857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:27.375857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:27.375857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:27.375857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:27.375857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:27.375857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:27.375857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:27.375857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:27.375857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 06 14:45:27.375857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_user(user_id) May 06 14:45:27.375857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 06 14:45:27.375857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 06 14:45:27.376880 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:27.376880 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:27.376880 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 06 14:45:27.376880 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:27.376880 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 06 14:45:27.376880 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:27.376880 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:27.376880 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:27.376880 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:27.376880 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:27.376880 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:27.376880 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:27.376880 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:27.376880 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:27.376880 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:27.376880 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:27.376880 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:27.376880 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:27.377824 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:27.377824 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:27.377824 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 06 14:45:27.377824 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 06 14:45:27.377824 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 06 14:45:27.377824 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 06 14:45:27.377824 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 06 14:45:27.377824 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 06 14:45:27.377824 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 06 14:45:27.377824 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:27.377824 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 402/802] 10.222.0.23 () {56 vars in 1093 bytes} [Fri May 6 14:45:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:27.377824 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de42a7b3-5bc3-4c68-818f-18665542b39d None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:27.385540 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2c4674-00ce-4a5e-9afa-43cdc7904711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:27.386216 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2c4674-00ce-4a5e-9afa-43cdc7904711 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:27.386500 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2c4674-00ce-4a5e-9afa-43cdc7904711 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:27.386771 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2c4674-00ce-4a5e-9afa-43cdc7904711 None admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:27.387566 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2c4674-00ce-4a5e-9afa-43cdc7904711 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:27.387876 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2c4674-00ce-4a5e-9afa-43cdc7904711 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:27.389341 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2c4674-00ce-4a5e-9afa-43cdc7904711 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:27.406554 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 401/803] 10.222.0.23 () {56 vars in 1148 bytes} [Fri May 6 14:45:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:27.406554 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2c4674-00ce-4a5e-9afa-43cdc7904711 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:27.485311 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de42a7b3-5bc3-4c68-818f-18665542b39d None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:27.499242 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2c4674-00ce-4a5e-9afa-43cdc7904711 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:27.523269 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa56e310-8944-4dae-a62b-b3e792846f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:27.526846 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56e310-8944-4dae-a62b-b3e792846f00 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:27.529477 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56e310-8944-4dae-a62b-b3e792846f00 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:27.531962 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56e310-8944-4dae-a62b-b3e792846f00 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:27.536434 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb1e282-75bf-4b85-87ae-3850097693e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:27.537472 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1e282-75bf-4b85-87ae-3850097693e4 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:27.537834 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1e282-75bf-4b85-87ae-3850097693e4 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:27.538115 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1e282-75bf-4b85-87ae-3850097693e4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:27.538830 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb1e282-75bf-4b85-87ae-3850097693e4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:27.540376 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb1e282-75bf-4b85-87ae-3850097693e4 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:27.544742 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa56e310-8944-4dae-a62b-b3e792846f00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:45:27.552837 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-fa56e310-8944-4dae-a62b-b3e792846f00 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 06 14:45:27.552837 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:27.552837 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:27.552837 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:27.552837 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:27.552837 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:27.552837 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:27.552837 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:27.552837 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:27.552837 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:27.552837 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:27.552837 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:27.552837 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:45:27.552837 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:45:27.552837 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:45:27.552837 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:45:27.553845 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:45:27.553845 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:45:27.553845 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:45:27.553845 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:45:27.553845 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:27.553845 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:27.553845 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:27.553845 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:27.553845 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:27.553845 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:27.553845 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:27.553845 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:27.553845 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:27.553845 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:27.553845 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:27.553845 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:27.554693 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:27.554693 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:27.554693 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:27.554693 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:27.554693 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:45:27.554693 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:45:27.554693 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:45:27.554693 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:45:27.554693 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:45:27.554693 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:45:27.554693 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 06 14:45:27.554693 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:27.560733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-bfb1e282-75bf-4b85-87ae-3850097693e4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 06 14:45:27.560733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:27.560733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 06 14:45:27.560733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 06 14:45:27.560733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 06 14:45:27.560733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 06 14:45:27.560733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 06 14:45:27.560733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:27.560733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 06 14:45:27.560733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:27.560733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 06 14:45:27.560733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:27.560733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:27.560733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:27.560733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:27.560733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:27.561592 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:27.561592 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:27.561592 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:27.561592 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:27.561592 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:27.561592 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:27.561592 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:27.561592 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 06 14:45:27.561592 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(service_id) May 06 14:45:27.561592 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 06 14:45:27.561592 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 06 14:45:27.561592 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:27.561592 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:27.561592 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:27.561592 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:27.561592 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:27.562429 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:27.562429 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:27.562429 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:27.562429 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:27.562429 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:27.562429 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:27.562429 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:27.562429 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:27.562429 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:27.562429 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:27.562429 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:27.562429 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 06 14:45:27.562429 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:27.562429 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 06 14:45:27.562429 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:27.563251 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 403/804] 10.222.0.23 () {56 vars in 1102 bytes} [Fri May 6 14:45:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:27.563323 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 402/805] 10.222.0.23 () {56 vars in 1129 bytes} [Fri May 6 14:45:27 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:27.563391 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa56e310-8944-4dae-a62b-b3e792846f00 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:27.563467 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb1e282-75bf-4b85-87ae-3850097693e4 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:27.658888 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb1e282-75bf-4b85-87ae-3850097693e4 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:27.659087 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa56e310-8944-4dae-a62b-b3e792846f00 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:27.691538 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f28f492-a92c-481b-9624-a659584f952a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:27.691997 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63535e16-3471-47b5-b948-0f1530e15c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:27.692745 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f28f492-a92c-481b-9624-a659584f952a None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:27.693424 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f28f492-a92c-481b-9624-a659584f952a None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:27.693524 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63535e16-3471-47b5-b948-0f1530e15c67 None admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:27.693742 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63535e16-3471-47b5-b948-0f1530e15c67 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:27.693837 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f28f492-a92c-481b-9624-a659584f952a None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:27.694018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63535e16-3471-47b5-b948-0f1530e15c67 None admin] PATH_INFO: `/v3/projects/d5398feee6eb490cb49bb261448a1a9b/users/bf9eed62c8f1409282c2af6488426a0f/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:27.694693 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f28f492-a92c-481b-9624-a659584f952a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:27.695004 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f28f492-a92c-481b-9624-a659584f952a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:27.696044 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63535e16-3471-47b5-b948-0f1530e15c67 None admin] RBAC: Authorizing `identity:create_grant(project_id=d5398feee6eb490cb49bb261448a1a9b, user_id=bf9eed62c8f1409282c2af6488426a0f, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:27.696422 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f28f492-a92c-481b-9624-a659584f952a None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:27.697483 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63535e16-3471-47b5-b948-0f1530e15c67 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:27.710923 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 403/806] 10.222.0.23 () {58 vars in 1439 bytes} [Fri May 6 14:45:27 2022] PUT /identity/v3/projects/d5398feee6eb490cb49bb261448a1a9b/users/bf9eed62c8f1409282c2af6488426a0f/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:45:27.712050 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63535e16-3471-47b5-b948-0f1530e15c67 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:27.735729 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 404/807] 10.222.0.23 () {56 vars in 1133 bytes} [Fri May 6 14:45:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:27.737557 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f28f492-a92c-481b-9624-a659584f952a None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:27.840550 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63535e16-3471-47b5-b948-0f1530e15c67 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:27.874395 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f28f492-a92c-481b-9624-a659584f952a None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:27.876885 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6513b25e-b675-407b-8ad8-21491a1d25d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:27.877697 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513b25e-b675-407b-8ad8-21491a1d25d0 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:27.878075 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513b25e-b675-407b-8ad8-21491a1d25d0 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:27.878445 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513b25e-b675-407b-8ad8-21491a1d25d0 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:27.879296 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6513b25e-b675-407b-8ad8-21491a1d25d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:27.879728 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6513b25e-b675-407b-8ad8-21491a1d25d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:27.881724 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6513b25e-b675-407b-8ad8-21491a1d25d0 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:27.890626 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 404/808] 10.222.0.23 () {56 vars in 1103 bytes} [Fri May 6 14:45:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:27.891634 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6513b25e-b675-407b-8ad8-21491a1d25d0 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:27.922377 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e2ac7a-161c-4812-b6a8-f2bd22df0d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:27.923191 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2ac7a-161c-4812-b6a8-f2bd22df0d61 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:27.923697 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2ac7a-161c-4812-b6a8-f2bd22df0d61 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:27.924214 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2ac7a-161c-4812-b6a8-f2bd22df0d61 None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:27.925088 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e2ac7a-161c-4812-b6a8-f2bd22df0d61 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:27.925570 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e2ac7a-161c-4812-b6a8-f2bd22df0d61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:27.928478 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e2ac7a-161c-4812-b6a8-f2bd22df0d61 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:27.934112 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 405/809] 10.222.0.23 () {56 vars in 1124 bytes} [Fri May 6 14:45:27 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:27.934827 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e2ac7a-161c-4812-b6a8-f2bd22df0d61 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:28.011875 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6513b25e-b675-407b-8ad8-21491a1d25d0 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:28.045245 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e2ac7a-161c-4812-b6a8-f2bd22df0d61 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:28.046618 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec7207-7c6b-4aa9-a15e-a27aef2ecfd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:28.047456 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec7207-7c6b-4aa9-a15e-a27aef2ecfd3 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:28.047885 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec7207-7c6b-4aa9-a15e-a27aef2ecfd3 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:28.048304 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec7207-7c6b-4aa9-a15e-a27aef2ecfd3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:28.081651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a455a-c068-4aed-becd-6e24f1c7c3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:28.082462 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a455a-c068-4aed-becd-6e24f1c7c3e3 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:28.082748 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a455a-c068-4aed-becd-6e24f1c7c3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:28.083123 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a455a-c068-4aed-becd-6e24f1c7c3e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:28.096213 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91a455a-c068-4aed-becd-6e24f1c7c3e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:28.098234 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91a455a-c068-4aed-becd-6e24f1c7c3e3 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:28.123279 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ec7207-7c6b-4aa9-a15e-a27aef2ecfd3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:28.124064 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-f91a455a-c068-4aed-becd-6e24f1c7c3e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:28.124064 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:28.124064 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:28.124064 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:28.124064 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:28.124064 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:28.124064 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:28.124064 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:28.124064 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:28.124064 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:28.124064 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:28.124064 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:28.124064 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:28.124064 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:28.124064 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:28.125190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:28.125190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:28.125190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:28.125190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:28.125190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:28.125190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:28.125190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:28.125190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:28.125190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:28.125190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:28.125190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:28.125190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:28.125190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:28.125190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:28.125190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:28.125190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:28.125190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:28.126341 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:28.126341 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:28.126341 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:28.126341 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:28.126341 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 406/810] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:28.126809 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a455a-c068-4aed-becd-6e24f1c7c3e3 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:28.128131 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ec7207-7c6b-4aa9-a15e-a27aef2ecfd3 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:28.241466 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a455a-c068-4aed-becd-6e24f1c7c3e3 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:28.260690 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 405/811] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:45:27 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 06 14:45:28.261848 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec7207-7c6b-4aa9-a15e-a27aef2ecfd3 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:28.281824 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d51a69-d00e-4c34-a9ff-4f5778aad8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:28.282862 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d51a69-d00e-4c34-a9ff-4f5778aad8d5 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:28.283345 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d51a69-d00e-4c34-a9ff-4f5778aad8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:28.283842 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d51a69-d00e-4c34-a9ff-4f5778aad8d5 None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:28.284781 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d51a69-d00e-4c34-a9ff-4f5778aad8d5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:28.285286 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d51a69-d00e-4c34-a9ff-4f5778aad8d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:28.287376 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d51a69-d00e-4c34-a9ff-4f5778aad8d5 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:28.293757 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 407/812] 10.222.0.23 () {56 vars in 1124 bytes} [Fri May 6 14:45:28 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:28.295273 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d51a69-d00e-4c34-a9ff-4f5778aad8d5 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:28.381799 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec7207-7c6b-4aa9-a15e-a27aef2ecfd3 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:28.403741 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d51a69-d00e-4c34-a9ff-4f5778aad8d5 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:28.416650 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f845f9-40bb-4011-8f96-db3c71d0723f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:28.417417 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f845f9-40bb-4011-8f96-db3c71d0723f None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:28.417683 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f845f9-40bb-4011-8f96-db3c71d0723f None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:28.417960 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f845f9-40bb-4011-8f96-db3c71d0723f None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:28.418671 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f845f9-40bb-4011-8f96-db3c71d0723f None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:28.418975 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f845f9-40bb-4011-8f96-db3c71d0723f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:28.420326 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f845f9-40bb-4011-8f96-db3c71d0723f None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:28.431188 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 406/813] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:28.431959 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f845f9-40bb-4011-8f96-db3c71d0723f None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:28.441870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344470eb-491a-4f71-9d54-95e741801b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:28.443369 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344470eb-491a-4f71-9d54-95e741801b60 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:28.443767 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344470eb-491a-4f71-9d54-95e741801b60 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:28.444076 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344470eb-491a-4f71-9d54-95e741801b60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:28.454697 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344470eb-491a-4f71-9d54-95e741801b60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:28.456129 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344470eb-491a-4f71-9d54-95e741801b60 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:28.473913 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-344470eb-491a-4f71-9d54-95e741801b60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:28.473913 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:28.473913 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:28.473913 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:28.473913 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:28.473913 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:28.473913 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:28.473913 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:28.473913 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:28.473913 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:28.473913 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:28.473913 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:28.473913 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:28.473913 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:28.473913 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:28.474858 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:28.474858 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:28.474858 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:28.474858 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:28.474858 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:28.474858 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:28.474858 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:28.474858 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:28.474858 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:28.474858 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:28.474858 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:28.474858 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:28.474858 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:28.474858 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:28.474858 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:28.474858 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:28.474858 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:28.475724 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:28.475724 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:28.475724 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:28.475724 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:28.475724 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 408/814] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:28.476566 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6508b98-0776-405b-8720-c090a65528c6 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:28.476566 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6508b98-0776-405b-8720-c090a65528c6 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:28.476832 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6508b98-0776-405b-8720-c090a65528c6 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:28.480776 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 409/815] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:45:28.480776 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6508b98-0776-405b-8720-c090a65528c6 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:28.541416 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f845f9-40bb-4011-8f96-db3c71d0723f None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:28.575485 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40da235e-82fb-4d06-9191-9118860cd9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:28.576138 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da235e-82fb-4d06-9191-9118860cd9c6 None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:28.576419 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da235e-82fb-4d06-9191-9118860cd9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:28.576721 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da235e-82fb-4d06-9191-9118860cd9c6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:28.577457 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40da235e-82fb-4d06-9191-9118860cd9c6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:28.578806 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40da235e-82fb-4d06-9191-9118860cd9c6 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:28.587145 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6508b98-0776-405b-8720-c090a65528c6 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:28.599625 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 407/816] 10.222.0.23 () {60 vars in 1144 bytes} [Fri May 6 14:45:28 2022] POST /identity/v3/endpoints => generated 346 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:45:28.600545 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40da235e-82fb-4d06-9191-9118860cd9c6 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:28.662986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3225622d-2261-4785-a20d-83704d85226d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:28.664073 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3225622d-2261-4785-a20d-83704d85226d None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:28.664500 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3225622d-2261-4785-a20d-83704d85226d None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:28.664977 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3225622d-2261-4785-a20d-83704d85226d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:28.678864 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3225622d-2261-4785-a20d-83704d85226d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:28.680501 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3225622d-2261-4785-a20d-83704d85226d None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:28.697384 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-3225622d-2261-4785-a20d-83704d85226d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 06 14:45:28.697384 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:28.697384 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:28.697384 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:28.697384 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:28.697384 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:28.697384 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:28.697384 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:28.697384 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:28.697384 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:28.697384 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:28.697384 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:28.697384 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 06 14:45:28.697384 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_user(user_id) May 06 14:45:28.697384 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 06 14:45:28.697384 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 06 14:45:28.698596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:28.698596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:28.698596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 06 14:45:28.698596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:28.698596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 06 14:45:28.698596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:28.698596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:28.698596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:28.698596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:28.698596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:28.698596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:28.698596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:28.698596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:28.698596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:28.698596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:28.698596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:28.698596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:28.698596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:28.699758 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:28.699758 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:28.699758 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 06 14:45:28.699758 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 06 14:45:28.699758 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 06 14:45:28.699758 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 06 14:45:28.699758 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 06 14:45:28.699758 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 06 14:45:28.699758 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 06 14:45:28.699758 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:28.701620 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 410/817] 10.222.0.23 () {56 vars in 1117 bytes} [Fri May 6 14:45:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:28.703228 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ca7b1-f56b-45ac-961b-d4af3b08d909 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:28.703652 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ca7b1-f56b-45ac-961b-d4af3b08d909 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:28.704026 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ca7b1-f56b-45ac-961b-d4af3b08d909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:28.723262 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-aa6ca7b1-f56b-45ac-961b-d4af3b08d909 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:28.724882 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-aa6ca7b1-f56b-45ac-961b-d4af3b08d909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:28.725235 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40da235e-82fb-4d06-9191-9118860cd9c6 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:28.765174 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae5c299-b057-458a-8a86-c120658db02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:28.766116 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5c299-b057-458a-8a86-c120658db02f None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:28.766394 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5c299-b057-458a-8a86-c120658db02f None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:28.766669 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5c299-b057-458a-8a86-c120658db02f None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:28.767396 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae5c299-b057-458a-8a86-c120658db02f None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:28.767700 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae5c299-b057-458a-8a86-c120658db02f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:28.769147 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae5c299-b057-458a-8a86-c120658db02f None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:28.788168 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 408/818] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:28.789193 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae5c299-b057-458a-8a86-c120658db02f None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:28.801514 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 411/819] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:45:28.803343 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6ca7b1-f56b-45ac-961b-d4af3b08d909 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:28.897389 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae5c299-b057-458a-8a86-c120658db02f None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:28.909524 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6ca7b1-f56b-45ac-961b-d4af3b08d909 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:28.937717 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170828e1-cf06-410b-8b87-a5f5405e38fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:28.938828 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170828e1-cf06-410b-8b87-a5f5405e38fe None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:28.939335 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170828e1-cf06-410b-8b87-a5f5405e38fe None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:28.939862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170828e1-cf06-410b-8b87-a5f5405e38fe None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:28.941084 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170828e1-cf06-410b-8b87-a5f5405e38fe None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:28.941648 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170828e1-cf06-410b-8b87-a5f5405e38fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:28.944442 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170828e1-cf06-410b-8b87-a5f5405e38fe None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:28.952573 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb1850-e987-4e71-8213-3f3960040b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:28.953656 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb1850-e987-4e71-8213-3f3960040b98 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:28.954174 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb1850-e987-4e71-8213-3f3960040b98 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:28.954610 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb1850-e987-4e71-8213-3f3960040b98 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:28.963342 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eb1850-e987-4e71-8213-3f3960040b98 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:28.965130 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eb1850-e987-4e71-8213-3f3960040b98 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:28.965810 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 409/820] 10.222.0.23 () {56 vars in 1149 bytes} [Fri May 6 14:45:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:28.966793 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170828e1-cf06-410b-8b87-a5f5405e38fe None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:28.982053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-12eb1850-e987-4e71-8213-3f3960040b98 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:28.982053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:28.982053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:28.982053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:28.982053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:28.982053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:28.982053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:28.982053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:28.982053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:28.982053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:28.982053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:28.982053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:28.982053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 06 14:45:28.982053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(project_id) May 06 14:45:28.982053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 06 14:45:28.983222 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 06 14:45:28.983222 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:28.983222 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:28.983222 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:28.983222 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:28.983222 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:28.983222 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:28.983222 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:28.983222 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:28.983222 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:28.983222 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:28.983222 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:28.983222 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:28.983222 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:28.983222 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:28.983222 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:28.983222 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:28.984327 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 06 14:45:28.984327 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 06 14:45:28.984327 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 06 14:45:28.984327 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 06 14:45:28.984327 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 06 14:45:28.984327 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 06 14:45:28.984327 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:28.984327 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:28.986722 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 412/821] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:45:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:28.992636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb1850-e987-4e71-8213-3f3960040b98 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:29.074608 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170828e1-cf06-410b-8b87-a5f5405e38fe None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:29.104130 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb1850-e987-4e71-8213-3f3960040b98 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:29.111193 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e25bea-e93a-4f6f-955a-dfc3d765ca77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:29.112267 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e25bea-e93a-4f6f-955a-dfc3d765ca77 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:29.112734 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e25bea-e93a-4f6f-955a-dfc3d765ca77 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:29.113166 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e25bea-e93a-4f6f-955a-dfc3d765ca77 None admin] PATH_INFO: `/v3/services/network` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:29.113991 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e25bea-e93a-4f6f-955a-dfc3d765ca77 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:29.116847 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e25bea-e93a-4f6f-955a-dfc3d765ca77 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:29.130030 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-c1e25bea-e93a-4f6f-955a-dfc3d765ca77 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 06 14:45:29.130030 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:29.130030 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 06 14:45:29.130030 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 06 14:45:29.130030 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 06 14:45:29.130030 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 06 14:45:29.130030 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 06 14:45:29.130030 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:29.130030 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 06 14:45:29.130030 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:29.130030 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 06 14:45:29.130030 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:29.130030 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:29.130030 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:29.130030 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:29.130030 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:29.130030 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:29.131833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:29.131833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:29.131833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:29.131833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:29.131833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:29.131833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:29.131833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 06 14:45:29.131833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(service_id) May 06 14:45:29.131833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 06 14:45:29.131833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 06 14:45:29.131833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:29.131833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:29.131833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:29.131833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:29.131833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:29.131833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:29.132883 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:29.132883 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:29.132883 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:29.132883 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:29.132883 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:29.132883 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:29.132883 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:29.132883 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:29.132883 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:29.132883 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:29.132883 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 06 14:45:29.132883 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:29.132883 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 06 14:45:29.132883 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:29.135496 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 410/822] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:45:28 2022] GET /identity/v3/services/network => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:29.141654 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e25bea-e93a-4f6f-955a-dfc3d765ca77 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:29.144943 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5826ced-5461-4e50-889f-b89ae9e358e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:29.146187 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5826ced-5461-4e50-889f-b89ae9e358e0 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:29.146794 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5826ced-5461-4e50-889f-b89ae9e358e0 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:29.147305 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5826ced-5461-4e50-889f-b89ae9e358e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:29.169865 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5826ced-5461-4e50-889f-b89ae9e358e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:29.171510 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5826ced-5461-4e50-889f-b89ae9e358e0 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:29.192246 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-d5826ced-5461-4e50-889f-b89ae9e358e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:29.192246 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:29.192246 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:29.192246 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:29.192246 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:29.192246 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:29.192246 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:29.192246 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:29.192246 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:29.192246 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:29.192246 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:29.192246 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:29.192246 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:29.192246 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:29.192246 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:29.193331 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:29.193331 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:29.193331 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:29.193331 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:29.193331 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:29.193331 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:29.193331 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:29.193331 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:29.193331 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:29.193331 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:29.193331 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:29.193331 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:29.193331 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:29.193331 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:29.193331 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:29.193331 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:29.193331 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:29.194288 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:29.194288 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:29.194288 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:29.194288 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:29.201189 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 413/823] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:29.201800 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5826ced-5461-4e50-889f-b89ae9e358e0 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:29.251151 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e25bea-e93a-4f6f-955a-dfc3d765ca77 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:29.290832 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8594975d-a065-44ba-9c1b-ce9148899e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:29.291494 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594975d-a065-44ba-9c1b-ce9148899e7b None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:29.291775 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594975d-a065-44ba-9c1b-ce9148899e7b None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:29.292047 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594975d-a065-44ba-9c1b-ce9148899e7b None admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:29.292852 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8594975d-a065-44ba-9c1b-ce9148899e7b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:29.293167 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8594975d-a065-44ba-9c1b-ce9148899e7b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:29.294735 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8594975d-a065-44ba-9c1b-ce9148899e7b None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:29.308329 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 411/824] 10.222.0.23 () {56 vars in 1148 bytes} [Fri May 6 14:45:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:29.309276 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8594975d-a065-44ba-9c1b-ce9148899e7b None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:29.324845 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5826ced-5461-4e50-889f-b89ae9e358e0 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:29.366990 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91673d0-4c01-4f1a-b86c-9be42c1a7a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:29.368097 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91673d0-4c01-4f1a-b86c-9be42c1a7a6a None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:29.368654 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91673d0-4c01-4f1a-b86c-9be42c1a7a6a None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:29.369130 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91673d0-4c01-4f1a-b86c-9be42c1a7a6a None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:29.370038 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91673d0-4c01-4f1a-b86c-9be42c1a7a6a None admin] RBAC: Adding query filter params (name=network) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:29.370554 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91673d0-4c01-4f1a-b86c-9be42c1a7a6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:29.372133 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91673d0-4c01-4f1a-b86c-9be42c1a7a6a None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:29.381966 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 414/825] 10.222.0.23 () {56 vars in 1112 bytes} [Fri May 6 14:45:29 2022] GET /identity/v3/services?name=network => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:29.383141 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91673d0-4c01-4f1a-b86c-9be42c1a7a6a None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:29.413941 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8594975d-a065-44ba-9c1b-ce9148899e7b None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:29.447791 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de894463-742e-4e71-a8af-92f8f7cdb78b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:29.448457 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de894463-742e-4e71-a8af-92f8f7cdb78b None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:29.448844 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de894463-742e-4e71-a8af-92f8f7cdb78b None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:29.449123 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de894463-742e-4e71-a8af-92f8f7cdb78b None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:29.450060 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de894463-742e-4e71-a8af-92f8f7cdb78b None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:29.450375 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de894463-742e-4e71-a8af-92f8f7cdb78b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:29.452269 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de894463-742e-4e71-a8af-92f8f7cdb78b None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:29.471262 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 412/826] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:29.471262 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de894463-742e-4e71-a8af-92f8f7cdb78b None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:29.491610 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91673d0-4c01-4f1a-b86c-9be42c1a7a6a None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:29.533751 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a63a7-62f7-407e-b94f-112aebb906b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:29.534977 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a63a7-62f7-407e-b94f-112aebb906b5 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:29.535539 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a63a7-62f7-407e-b94f-112aebb906b5 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:29.536733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a63a7-62f7-407e-b94f-112aebb906b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:29.538085 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9a63a7-62f7-407e-b94f-112aebb906b5 None admin] RBAC: Adding query filter params (role.id=6dcd35d1a83748759d6ef9f026077835, scope.project.id=d5398feee6eb490cb49bb261448a1a9b, user.id=fc8cf09642ec4e10a80badcf8b585ef8) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:29.538683 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9a63a7-62f7-407e-b94f-112aebb906b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:29.540690 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9a63a7-62f7-407e-b94f-112aebb906b5 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:29.548650 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 415/827] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:45:29 2022] GET /identity/v3/role_assignments?user.id=fc8cf09642ec4e10a80badcf8b585ef8&scope.project.id=d5398feee6eb490cb49bb261448a1a9b&role.id=6dcd35d1a83748759d6ef9f026077835 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:29.550105 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a63a7-62f7-407e-b94f-112aebb906b5 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:29.597562 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de894463-742e-4e71-a8af-92f8f7cdb78b None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:29.641053 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9284578-0e70-4ba2-9fce-c1d26531c2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:29.641742 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9284578-0e70-4ba2-9fce-c1d26531c2c1 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:29.642026 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9284578-0e70-4ba2-9fce-c1d26531c2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:29.642304 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9284578-0e70-4ba2-9fce-c1d26531c2c1 None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:29.643923 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9284578-0e70-4ba2-9fce-c1d26531c2c1 None admin] RBAC: Adding query filter params (type=network) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:29.644241 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9284578-0e70-4ba2-9fce-c1d26531c2c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:29.645941 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9284578-0e70-4ba2-9fce-c1d26531c2c1 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:29.652119 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 413/828] 10.222.0.23 () {56 vars in 1112 bytes} [Fri May 6 14:45:29 2022] GET /identity/v3/services?type=network => generated 350 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:29.657858 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9284578-0e70-4ba2-9fce-c1d26531c2c1 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:29.703138 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a63a7-62f7-407e-b94f-112aebb906b5 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:29.774704 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9284578-0e70-4ba2-9fce-c1d26531c2c1 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:29.779180 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b8d100-9720-48c9-a964-4a6fbe2eaf67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:29.780221 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8d100-9720-48c9-a964-4a6fbe2eaf67 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:29.781000 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8d100-9720-48c9-a964-4a6fbe2eaf67 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:29.781548 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8d100-9720-48c9-a964-4a6fbe2eaf67 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:29.800188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b8d100-9720-48c9-a964-4a6fbe2eaf67 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:29.802688 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b8d100-9720-48c9-a964-4a6fbe2eaf67 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:29.810823 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b22fa11-4a9e-4ef3-8364-cb84930d5624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:29.811491 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22fa11-4a9e-4ef3-8364-cb84930d5624 None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:29.811778 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22fa11-4a9e-4ef3-8364-cb84930d5624 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:29.812053 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22fa11-4a9e-4ef3-8364-cb84930d5624 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:29.812835 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b22fa11-4a9e-4ef3-8364-cb84930d5624 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:29.814201 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b22fa11-4a9e-4ef3-8364-cb84930d5624 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:29.828049 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-74b8d100-9720-48c9-a964-4a6fbe2eaf67 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:29.828049 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:29.828049 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:29.828049 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:29.828049 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:29.828049 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:29.828049 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:29.828049 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:29.828049 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:29.828049 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:29.828049 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:29.828049 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:29.828049 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 06 14:45:29.828049 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(project_id) May 06 14:45:29.828049 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 06 14:45:29.829034 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 06 14:45:29.829034 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:29.829034 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:29.829034 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:29.829034 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:29.829034 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:29.829034 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:29.829034 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:29.829034 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:29.829034 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:29.829034 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:29.829034 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:29.829034 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:29.829034 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:29.829034 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:29.829034 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:29.829034 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:29.830150 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 06 14:45:29.830150 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 06 14:45:29.830150 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 06 14:45:29.830150 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 06 14:45:29.830150 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 06 14:45:29.830150 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 06 14:45:29.830150 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:29.830150 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:29.836980 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 416/829] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:45:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:29.839252 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b8d100-9720-48c9-a964-4a6fbe2eaf67 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:29.843770 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 414/830] 10.222.0.23 () {60 vars in 1144 bytes} [Fri May 6 14:45:29 2022] POST /identity/v3/endpoints => generated 327 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 14:45:30.000709 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b8d100-9720-48c9-a964-4a6fbe2eaf67 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:30.047196 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ab76af-89fc-41ad-a0c1-707f95443fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:30.047872 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab76af-89fc-41ad-a0c1-707f95443fb2 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:30.048152 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab76af-89fc-41ad-a0c1-707f95443fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:30.048424 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab76af-89fc-41ad-a0c1-707f95443fb2 None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:30.049283 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ab76af-89fc-41ad-a0c1-707f95443fb2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:30.049597 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ab76af-89fc-41ad-a0c1-707f95443fb2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:30.051085 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ab76af-89fc-41ad-a0c1-707f95443fb2 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:30.065202 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 417/831] 10.222.0.23 () {56 vars in 1148 bytes} [Fri May 6 14:45:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:30.072544 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b22fa11-4a9e-4ef3-8364-cb84930d5624 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:30.162722 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ecbc50-a812-4ffc-9612-75f417cd2c50 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:30.163036 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ecbc50-a812-4ffc-9612-75f417cd2c50 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:30.163332 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ecbc50-a812-4ffc-9612-75f417cd2c50 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:30.165115 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 418/832] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:45:30.171675 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada0e89-2fd8-48ce-b478-eb4a3ab55cfa None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:30.171964 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada0e89-2fd8-48ce-b478-eb4a3ab55cfa None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:30.172243 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada0e89-2fd8-48ce-b478-eb4a3ab55cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:30.195195 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-6ada0e89-2fd8-48ce-b478-eb4a3ab55cfa None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:30.196383 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-6ada0e89-2fd8-48ce-b478-eb4a3ab55cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:30.202934 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b22fa11-4a9e-4ef3-8364-cb84930d5624 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:30.245219 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5ab7d-b8b9-424e-8b31-ad1aa67125a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:30.246955 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5ab7d-b8b9-424e-8b31-ad1aa67125a4 None admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:30.247239 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5ab7d-b8b9-424e-8b31-ad1aa67125a4 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:30.247514 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5ab7d-b8b9-424e-8b31-ad1aa67125a4 None admin] PATH_INFO: `/v3/projects/d5398feee6eb490cb49bb261448a1a9b/users/a7dc1bc53c59423e934f7831ea022c73/roles/f5641817ee56438496afc6135d37a4c0` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:30.265813 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b5ab7d-b8b9-424e-8b31-ad1aa67125a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=d5398feee6eb490cb49bb261448a1a9b, user_id=a7dc1bc53c59423e934f7831ea022c73, role_id=f5641817ee56438496afc6135d37a4c0)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:30.267178 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b5ab7d-b8b9-424e-8b31-ad1aa67125a4 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:30.284084 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 419/833] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:30 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 06 14:45:30.284404 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 415/834] 10.222.0.23 () {58 vars in 1439 bytes} [Fri May 6 14:45:30 2022] PUT /identity/v3/projects/d5398feee6eb490cb49bb261448a1a9b/users/a7dc1bc53c59423e934f7831ea022c73/roles/f5641817ee56438496afc6135d37a4c0 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:45:30.309574 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada0e89-2fd8-48ce-b478-eb4a3ab55cfa None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:30.433542 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada0e89-2fd8-48ce-b478-eb4a3ab55cfa None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:30.472590 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6549014d-d16a-49a3-9976-3a5d87e4d573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:30.473896 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549014d-d16a-49a3-9976-3a5d87e4d573 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:30.474366 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549014d-d16a-49a3-9976-3a5d87e4d573 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:30.474796 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549014d-d16a-49a3-9976-3a5d87e4d573 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:30.486717 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6549014d-d16a-49a3-9976-3a5d87e4d573 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:45:30.495158 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-6549014d-d16a-49a3-9976-3a5d87e4d573 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:45:30.495158 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:30.495158 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:30.495158 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:30.495158 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:30.495158 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:30.495158 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:30.495158 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:30.495158 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:30.495158 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:30.495158 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:30.495158 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:30.495158 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:45:30.495158 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:45:30.495158 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:45:30.495158 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:45:30.496351 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:45:30.496351 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:45:30.496351 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:45:30.496351 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:45:30.496351 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:30.496351 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:30.496351 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:30.496351 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:30.496351 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:30.496351 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:30.496351 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:30.496351 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:30.496351 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:30.496351 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:30.496351 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:30.496351 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:30.497541 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:30.497541 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:30.497541 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:30.497541 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:30.497541 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:45:30.497541 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:45:30.497541 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:45:30.497541 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:45:30.497541 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:45:30.497541 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:45:30.497541 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:45:30.497541 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:30.499762 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 420/835] 10.222.0.23 () {56 vars in 1096 bytes} [Fri May 6 14:45:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:30.506268 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5ab7d-b8b9-424e-8b31-ad1aa67125a4 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:30.620543 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5ab7d-b8b9-424e-8b31-ad1aa67125a4 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:30.656103 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edfabba-d500-4159-8344-ff19d55ea82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:30.656811 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfabba-d500-4159-8344-ff19d55ea82d None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:30.657094 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfabba-d500-4159-8344-ff19d55ea82d None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:30.657369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfabba-d500-4159-8344-ff19d55ea82d None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:30.658106 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edfabba-d500-4159-8344-ff19d55ea82d None admin] RBAC: Adding query filter params (name=admin) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:30.658410 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edfabba-d500-4159-8344-ff19d55ea82d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:30.659759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edfabba-d500-4159-8344-ff19d55ea82d None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:30.669000 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 416/836] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:45:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:30.674197 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6549014d-d16a-49a3-9976-3a5d87e4d573 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:30.788479 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6549014d-d16a-49a3-9976-3a5d87e4d573 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:30.823468 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d1b551-c4dc-481a-b415-79bc013319f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:30.824266 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d1b551-c4dc-481a-b415-79bc013319f9 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:30.824546 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d1b551-c4dc-481a-b415-79bc013319f9 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:30.824846 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d1b551-c4dc-481a-b415-79bc013319f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:30.837559 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d1b551-c4dc-481a-b415-79bc013319f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:30.839066 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d1b551-c4dc-481a-b415-79bc013319f9 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:30.857188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-37d1b551-c4dc-481a-b415-79bc013319f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:30.857188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:30.857188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:30.857188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:30.857188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:30.857188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:30.857188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:30.857188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:30.857188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:30.857188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:30.857188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:30.857188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:30.857188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:30.857188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:30.857188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:30.858152 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:30.858152 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:30.858152 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:30.858152 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:30.858152 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:30.858152 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:30.858152 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:30.858152 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:30.858152 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:30.858152 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:30.858152 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:30.858152 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:30.858152 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:30.858152 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:30.858152 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:30.858152 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:30.858152 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:30.859053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:30.859053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:30.859053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:30.859053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:30.859053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 421/837] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:30.864087 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edfabba-d500-4159-8344-ff19d55ea82d None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:30.970351 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edfabba-d500-4159-8344-ff19d55ea82d None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:31.006634 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0627be-2045-4f54-beb4-b8b5d459222d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:31.007296 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0627be-2045-4f54-beb4-b8b5d459222d None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:31.007576 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0627be-2045-4f54-beb4-b8b5d459222d None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:31.007846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0627be-2045-4f54-beb4-b8b5d459222d None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:31.008560 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0627be-2045-4f54-beb4-b8b5d459222d None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:31.008894 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0627be-2045-4f54-beb4-b8b5d459222d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:31.010247 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0627be-2045-4f54-beb4-b8b5d459222d None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:31.023631 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 417/838] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:31.029683 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d1b551-c4dc-481a-b415-79bc013319f9 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:31.142630 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d1b551-c4dc-481a-b415-79bc013319f9 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:31.179884 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20cc0ca-1fde-4fd6-af90-59f9b8dad6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:31.180709 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20cc0ca-1fde-4fd6-af90-59f9b8dad6a0 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:31.180991 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20cc0ca-1fde-4fd6-af90-59f9b8dad6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:31.181267 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20cc0ca-1fde-4fd6-af90-59f9b8dad6a0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:31.189131 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20cc0ca-1fde-4fd6-af90-59f9b8dad6a0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:31.190562 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20cc0ca-1fde-4fd6-af90-59f9b8dad6a0 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:31.205688 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-f20cc0ca-1fde-4fd6-af90-59f9b8dad6a0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 06 14:45:31.205688 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:31.205688 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:31.205688 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:31.205688 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:31.205688 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:31.205688 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:31.205688 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:31.205688 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:31.205688 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:31.205688 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:31.205688 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:31.205688 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 06 14:45:31.205688 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_user(user_id) May 06 14:45:31.205688 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 06 14:45:31.205688 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 06 14:45:31.206698 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:31.206698 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:31.206698 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 06 14:45:31.206698 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:31.206698 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 06 14:45:31.206698 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:31.206698 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:31.206698 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:31.206698 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:31.206698 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:31.206698 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:31.206698 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:31.206698 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:31.206698 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:31.206698 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:31.206698 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:31.206698 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:31.206698 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:31.208463 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:31.208463 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:31.208463 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 06 14:45:31.208463 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 06 14:45:31.208463 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 06 14:45:31.208463 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 06 14:45:31.208463 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 06 14:45:31.208463 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 06 14:45:31.208463 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 06 14:45:31.208463 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:31.208463 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 422/839] 10.222.0.23 () {56 vars in 1096 bytes} [Fri May 6 14:45:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:31.216154 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0627be-2045-4f54-beb4-b8b5d459222d None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:31.316530 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0627be-2045-4f54-beb4-b8b5d459222d None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:31.335763 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f49db2-0914-429d-8b86-eda084b2731d None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:31.336048 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f49db2-0914-429d-8b86-eda084b2731d None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:31.336321 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f49db2-0914-429d-8b86-eda084b2731d None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:31.338027 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 423/840] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:45:31.343178 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3789d-4e67-4a37-9278-7084562ce58c None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:31.343478 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3789d-4e67-4a37-9278-7084562ce58c None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:31.343764 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3789d-4e67-4a37-9278-7084562ce58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:31.355416 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d0210-0ea9-4f70-b960-e19423065cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:31.356441 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d0210-0ea9-4f70-b960-e19423065cf1 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:31.356964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d0210-0ea9-4f70-b960-e19423065cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:31.357444 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d0210-0ea9-4f70-b960-e19423065cf1 None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:31.358487 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d0210-0ea9-4f70-b960-e19423065cf1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:31.359039 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d0210-0ea9-4f70-b960-e19423065cf1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:31.360720 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d0210-0ea9-4f70-b960-e19423065cf1 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:31.361707 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-2ef3789d-4e67-4a37-9278-7084562ce58c None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:31.362947 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-2ef3789d-4e67-4a37-9278-7084562ce58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:31.384781 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 418/841] 10.222.0.23 () {56 vars in 1135 bytes} [Fri May 6 14:45:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:31.386459 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d0210-0ea9-4f70-b960-e19423065cf1 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:31.444715 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 424/842] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:31 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 06 14:45:31.471703 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef3789d-4e67-4a37-9278-7084562ce58c None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:31.490680 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d0210-0ea9-4f70-b960-e19423065cf1 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:31.528883 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a002be-9e7c-4621-a8f2-06faf0c00b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:31.529972 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a002be-9e7c-4621-a8f2-06faf0c00b43 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:31.530752 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a002be-9e7c-4621-a8f2-06faf0c00b43 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:31.531232 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a002be-9e7c-4621-a8f2-06faf0c00b43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:31.576515 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef3789d-4e67-4a37-9278-7084562ce58c None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:31.603400 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a002be-9e7c-4621-a8f2-06faf0c00b43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:31.605228 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a002be-9e7c-4621-a8f2-06faf0c00b43 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:31.615156 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae9ba5-e6d2-4249-a073-4cd463db5fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:31.615970 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae9ba5-e6d2-4249-a073-4cd463db5fb4 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:31.616253 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae9ba5-e6d2-4249-a073-4cd463db5fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:31.616531 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae9ba5-e6d2-4249-a073-4cd463db5fb4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:31.617589 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acae9ba5-e6d2-4249-a073-4cd463db5fb4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:31.618594 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acae9ba5-e6d2-4249-a073-4cd463db5fb4 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:31.632347 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-acae9ba5-e6d2-4249-a073-4cd463db5fb4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 06 14:45:31.632347 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:31.632347 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 06 14:45:31.632347 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 06 14:45:31.632347 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 06 14:45:31.632347 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 06 14:45:31.632347 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 06 14:45:31.632347 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:31.632347 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 06 14:45:31.632347 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:31.632347 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 06 14:45:31.632347 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:31.632347 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:31.632347 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:31.632347 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:31.632347 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:31.632347 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:31.633433 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:31.633433 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:31.633433 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:31.633433 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:31.633433 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:31.633433 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:31.633433 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 06 14:45:31.633433 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(service_id) May 06 14:45:31.633433 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 06 14:45:31.633433 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 06 14:45:31.633433 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:31.633433 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:31.633433 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:31.633433 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:31.633433 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:31.633433 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:31.634282 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:31.634282 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:31.634282 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:31.634282 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:31.634282 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:31.634282 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:31.634282 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:31.634282 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:31.634282 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:31.634282 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:31.634282 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 06 14:45:31.634282 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:31.634282 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 06 14:45:31.634282 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:31.638588 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 425/843] 10.222.0.23 () {56 vars in 1114 bytes} [Fri May 6 14:45:31 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:45:31.639466 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae9ba5-e6d2-4249-a073-4cd463db5fb4 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:31.722341 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 419/844] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:45:31 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 06 14:45:31.723635 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a002be-9e7c-4621-a8f2-06faf0c00b43 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:31.742545 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae9ba5-e6d2-4249-a073-4cd463db5fb4 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:31.776728 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c425526-be55-4f4f-9cf3-d2510ff8ef42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:31.777402 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c425526-be55-4f4f-9cf3-d2510ff8ef42 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:31.777687 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c425526-be55-4f4f-9cf3-d2510ff8ef42 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:31.777965 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c425526-be55-4f4f-9cf3-d2510ff8ef42 None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:31.778839 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c425526-be55-4f4f-9cf3-d2510ff8ef42 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:31.778991 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c425526-be55-4f4f-9cf3-d2510ff8ef42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:31.780322 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c425526-be55-4f4f-9cf3-d2510ff8ef42 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:31.790120 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 426/845] 10.222.0.23 () {56 vars in 1114 bytes} [Fri May 6 14:45:31 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:31.790969 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c425526-be55-4f4f-9cf3-d2510ff8ef42 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:31.827963 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a002be-9e7c-4621-a8f2-06faf0c00b43 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:31.862558 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9d8d61-91ba-42d7-8ca6-6bd792fd75a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:31.863719 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d8d61-91ba-42d7-8ca6-6bd792fd75a3 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:31.864919 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d8d61-91ba-42d7-8ca6-6bd792fd75a3 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:31.865418 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d8d61-91ba-42d7-8ca6-6bd792fd75a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:31.883576 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9d8d61-91ba-42d7-8ca6-6bd792fd75a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:31.885365 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9d8d61-91ba-42d7-8ca6-6bd792fd75a3 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:31.892084 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c425526-be55-4f4f-9cf3-d2510ff8ef42 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:31.903797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-6d9d8d61-91ba-42d7-8ca6-6bd792fd75a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:31.903797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:31.903797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:31.903797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:31.903797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:31.903797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:31.903797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:31.903797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:31.903797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:31.903797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:31.903797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:31.903797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:31.903797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:31.903797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:31.903797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:31.904765 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:31.904765 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:31.904765 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:31.904765 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:31.904765 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:31.904765 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:31.904765 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:31.904765 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:31.904765 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:31.904765 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:31.904765 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:31.904765 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:31.904765 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:31.904765 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:31.904765 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:31.904765 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:31.904765 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:31.905944 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:31.905944 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:31.905944 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:31.905944 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:31.913147 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 420/846] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:31.914334 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9d8d61-91ba-42d7-8ca6-6bd792fd75a3 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:31.928028 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d44a3e-4e0a-44db-a105-a8165688094b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:31.928730 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d44a3e-4e0a-44db-a105-a8165688094b None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:31.929019 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d44a3e-4e0a-44db-a105-a8165688094b None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:31.929300 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d44a3e-4e0a-44db-a105-a8165688094b None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:31.930009 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d44a3e-4e0a-44db-a105-a8165688094b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:31.930316 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d44a3e-4e0a-44db-a105-a8165688094b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:31.931688 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d44a3e-4e0a-44db-a105-a8165688094b None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:31.936743 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 427/847] 10.222.0.23 () {56 vars in 1114 bytes} [Fri May 6 14:45:31 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:32.031178 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9d8d61-91ba-42d7-8ca6-6bd792fd75a3 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:32.069663 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a5bbb3-2dc5-431d-b1c4-a7b9d3954dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:32.070773 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5bbb3-2dc5-431d-b1c4-a7b9d3954dfa None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:32.071282 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5bbb3-2dc5-431d-b1c4-a7b9d3954dfa None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:32.071765 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5bbb3-2dc5-431d-b1c4-a7b9d3954dfa None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:32.072753 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a5bbb3-2dc5-431d-b1c4-a7b9d3954dfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:32.073280 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a5bbb3-2dc5-431d-b1c4-a7b9d3954dfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:32.074891 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a5bbb3-2dc5-431d-b1c4-a7b9d3954dfa None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:32.089466 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 421/848] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:32.097751 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d44a3e-4e0a-44db-a105-a8165688094b None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:32.223859 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d44a3e-4e0a-44db-a105-a8165688094b None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:32.259674 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ff3f1-33dc-46f3-9ddb-a6e163138d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:32.260886 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ff3f1-33dc-46f3-9ddb-a6e163138d76 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:32.261368 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ff3f1-33dc-46f3-9ddb-a6e163138d76 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:32.261864 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ff3f1-33dc-46f3-9ddb-a6e163138d76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:32.268989 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0ff3f1-33dc-46f3-9ddb-a6e163138d76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:32.271898 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0ff3f1-33dc-46f3-9ddb-a6e163138d76 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:32.287836 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-6e0ff3f1-33dc-46f3-9ddb-a6e163138d76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:32.287836 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:32.287836 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:32.287836 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:32.287836 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:32.287836 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:32.287836 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:32.287836 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:32.287836 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:32.287836 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:32.287836 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:32.287836 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:32.287836 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 06 14:45:32.287836 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(project_id) May 06 14:45:32.287836 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 06 14:45:32.288882 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 06 14:45:32.288882 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:32.288882 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:32.288882 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:32.288882 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:32.288882 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:32.288882 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:32.288882 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:32.288882 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:32.288882 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:32.288882 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:32.288882 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:32.288882 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:32.288882 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:32.288882 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:32.288882 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:32.288882 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:32.289774 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 06 14:45:32.289774 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 06 14:45:32.289774 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 06 14:45:32.289774 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 06 14:45:32.289774 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 06 14:45:32.289774 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 06 14:45:32.289774 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:32.289774 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:32.292346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 428/849] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:45:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:32.298472 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a5bbb3-2dc5-431d-b1c4-a7b9d3954dfa None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:32.333498 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ca3ac-aeca-4d18-9a0f-7ec3f0bac929 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:32.334020 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ca3ac-aeca-4d18-9a0f-7ec3f0bac929 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:32.334503 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ca3ac-aeca-4d18-9a0f-7ec3f0bac929 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:32.340006 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 429/850] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:45:32.341842 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf11e9-ac32-4863-a106-45eb57acfcea None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:32.342179 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf11e9-ac32-4863-a106-45eb57acfcea None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:32.342507 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf11e9-ac32-4863-a106-45eb57acfcea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:32.363102 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-adaf11e9-ac32-4863-a106-45eb57acfcea None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:32.365316 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-adaf11e9-ac32-4863-a106-45eb57acfcea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:32.433843 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a5bbb3-2dc5-431d-b1c4-a7b9d3954dfa None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:32.444753 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 430/851] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:32 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:45:32.459706 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaf11e9-ac32-4863-a106-45eb57acfcea None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:32.475820 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46989b6b-5c99-4d20-9239-fc953fb4377b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:32.477375 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46989b6b-5c99-4d20-9239-fc953fb4377b None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:32.478131 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46989b6b-5c99-4d20-9239-fc953fb4377b None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:32.478627 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46989b6b-5c99-4d20-9239-fc953fb4377b None admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:32.479831 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46989b6b-5c99-4d20-9239-fc953fb4377b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:32.480741 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46989b6b-5c99-4d20-9239-fc953fb4377b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:32.482438 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46989b6b-5c99-4d20-9239-fc953fb4377b None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:32.500115 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 422/852] 10.222.0.23 () {56 vars in 1148 bytes} [Fri May 6 14:45:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:32.507139 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46989b6b-5c99-4d20-9239-fc953fb4377b None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:32.583693 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaf11e9-ac32-4863-a106-45eb57acfcea None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:32.610326 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46989b6b-5c99-4d20-9239-fc953fb4377b None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:32.620711 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d21797f-5550-48d9-a3b5-b5ea2bb4c0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:32.621683 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21797f-5550-48d9-a3b5-b5ea2bb4c0a9 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:32.622027 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21797f-5550-48d9-a3b5-b5ea2bb4c0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:32.622357 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21797f-5550-48d9-a3b5-b5ea2bb4c0a9 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:32.623005 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d21797f-5550-48d9-a3b5-b5ea2bb4c0a9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:32.624445 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d21797f-5550-48d9-a3b5-b5ea2bb4c0a9 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:32.637734 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-7d21797f-5550-48d9-a3b5-b5ea2bb4c0a9 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 06 14:45:32.637734 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:32.637734 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 06 14:45:32.637734 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 06 14:45:32.637734 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 06 14:45:32.637734 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 06 14:45:32.637734 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 06 14:45:32.637734 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:32.637734 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 06 14:45:32.637734 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:32.637734 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 06 14:45:32.637734 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:32.637734 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:32.637734 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:32.637734 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:32.637734 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:32.638736 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:32.638736 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:32.638736 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:32.638736 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:32.638736 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:32.638736 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:32.638736 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:32.638736 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 06 14:45:32.638736 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(service_id) May 06 14:45:32.638736 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 06 14:45:32.638736 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 06 14:45:32.638736 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:32.638736 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:32.638736 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:32.638736 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:32.638736 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:32.639581 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:32.639581 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:32.639581 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:32.639581 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:32.639581 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:32.639581 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:32.639581 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:32.639581 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:32.639581 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:32.639581 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:32.639581 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:32.639581 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 06 14:45:32.639581 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:32.639581 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 06 14:45:32.639581 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:32.644121 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 431/853] 10.222.0.23 () {56 vars in 1132 bytes} [Fri May 6 14:45:32 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:32.645470 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d21797f-5550-48d9-a3b5-b5ea2bb4c0a9 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:32.650942 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769af667-5db2-457d-9f78-6e139d167984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:32.651890 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769af667-5db2-457d-9f78-6e139d167984 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:32.652380 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769af667-5db2-457d-9f78-6e139d167984 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:32.652882 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769af667-5db2-457d-9f78-6e139d167984 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:32.653915 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769af667-5db2-457d-9f78-6e139d167984 None admin] RBAC: Adding query filter params (role.id=6dcd35d1a83748759d6ef9f026077835, scope.project.id=d5398feee6eb490cb49bb261448a1a9b, user.id=bf9eed62c8f1409282c2af6488426a0f) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:32.654479 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769af667-5db2-457d-9f78-6e139d167984 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:32.656375 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769af667-5db2-457d-9f78-6e139d167984 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:32.668825 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 423/854] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:45:32 2022] GET /identity/v3/role_assignments?user.id=bf9eed62c8f1409282c2af6488426a0f&scope.project.id=d5398feee6eb490cb49bb261448a1a9b&role.id=6dcd35d1a83748759d6ef9f026077835 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:32.670322 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0c486-75e5-49a2-8d80-1e328377102b None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:32.670615 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0c486-75e5-49a2-8d80-1e328377102b None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:32.670897 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0c486-75e5-49a2-8d80-1e328377102b None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:32.672971 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 424/855] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:45:32.679348 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76330c-8ed1-418b-8b01-e945491b49ca None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:32.679647 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76330c-8ed1-418b-8b01-e945491b49ca None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:32.679920 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76330c-8ed1-418b-8b01-e945491b49ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:32.699738 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-dc76330c-8ed1-418b-8b01-e945491b49ca None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:32.701193 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-dc76330c-8ed1-418b-8b01-e945491b49ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:32.772508 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d21797f-5550-48d9-a3b5-b5ea2bb4c0a9 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:32.789049 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 425/856] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:32 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 06 14:45:32.805492 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76330c-8ed1-418b-8b01-e945491b49ca None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:32.816146 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af437134-943b-4576-a1e8-84a56494df42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:32.817639 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af437134-943b-4576-a1e8-84a56494df42 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:32.817912 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af437134-943b-4576-a1e8-84a56494df42 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:32.818176 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af437134-943b-4576-a1e8-84a56494df42 None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:32.818998 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af437134-943b-4576-a1e8-84a56494df42 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:32.819303 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af437134-943b-4576-a1e8-84a56494df42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:32.820883 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af437134-943b-4576-a1e8-84a56494df42 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:32.831402 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 432/857] 10.222.0.23 () {56 vars in 1126 bytes} [Fri May 6 14:45:32 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:32.833545 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af437134-943b-4576-a1e8-84a56494df42 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:32.933900 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76330c-8ed1-418b-8b01-e945491b49ca None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:32.963045 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af437134-943b-4576-a1e8-84a56494df42 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:32.979510 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf11dc5-e005-445e-b5c3-a2f184c39e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:32.980628 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf11dc5-e005-445e-b5c3-a2f184c39e92 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:32.980918 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf11dc5-e005-445e-b5c3-a2f184c39e92 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:32.981193 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf11dc5-e005-445e-b5c3-a2f184c39e92 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:32.995878 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf11dc5-e005-445e-b5c3-a2f184c39e92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:45:32.998245 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfda9eb-8461-44e6-b6bb-dac537055619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:32.998957 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfda9eb-8461-44e6-b6bb-dac537055619 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:32.999236 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfda9eb-8461-44e6-b6bb-dac537055619 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:32.999504 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfda9eb-8461-44e6-b6bb-dac537055619 None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:33.000243 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfda9eb-8461-44e6-b6bb-dac537055619 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:33.000549 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfda9eb-8461-44e6-b6bb-dac537055619 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:33.001925 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfda9eb-8461-44e6-b6bb-dac537055619 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:33.004044 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-1cf11dc5-e005-445e-b5c3-a2f184c39e92 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 06 14:45:33.004044 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:33.004044 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:33.004044 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:33.004044 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:33.004044 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:33.004044 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:33.004044 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:33.004044 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:33.004044 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:33.004044 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:33.004044 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:33.004044 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:45:33.004044 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:45:33.004044 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:45:33.004044 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:45:33.005596 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:45:33.005596 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:45:33.005596 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:45:33.005596 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:45:33.005596 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:33.005596 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:33.005596 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:33.005596 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:33.005596 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:33.005596 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:33.005596 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:33.005596 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:33.005596 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:33.005596 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:33.005596 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:33.005596 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:33.006575 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:33.006575 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:33.006575 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:33.006575 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:33.006575 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:45:33.006575 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:45:33.006575 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:45:33.006575 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:45:33.006575 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:45:33.006575 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:45:33.006575 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 06 14:45:33.006575 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:33.006575 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 426/858] 10.222.0.23 () {56 vars in 1102 bytes} [Fri May 6 14:45:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:45:33.008741 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 433/859] 10.222.0.23 () {56 vars in 1126 bytes} [Fri May 6 14:45:32 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:33.013104 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf11dc5-e005-445e-b5c3-a2f184c39e92 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:33.135369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf11dc5-e005-445e-b5c3-a2f184c39e92 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:33.188230 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdc529a-33b7-47fe-a3b7-7ea2f8d0e840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:33.188869 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc529a-33b7-47fe-a3b7-7ea2f8d0e840 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:33.189184 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc529a-33b7-47fe-a3b7-7ea2f8d0e840 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:33.189497 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc529a-33b7-47fe-a3b7-7ea2f8d0e840 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:33.190295 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdc529a-33b7-47fe-a3b7-7ea2f8d0e840 None admin] RBAC: Adding query filter params (name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:33.190606 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdc529a-33b7-47fe-a3b7-7ea2f8d0e840 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:33.192070 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdc529a-33b7-47fe-a3b7-7ea2f8d0e840 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:33.202235 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 427/860] 10.222.0.23 () {56 vars in 1103 bytes} [Fri May 6 14:45:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:33.207490 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfda9eb-8461-44e6-b6bb-dac537055619 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:33.319961 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfda9eb-8461-44e6-b6bb-dac537055619 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:33.355664 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e858eb-5c4b-4782-9d1c-33e512941c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:33.356546 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e858eb-5c4b-4782-9d1c-33e512941c2a None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:33.356878 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e858eb-5c4b-4782-9d1c-33e512941c2a None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:33.357199 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e858eb-5c4b-4782-9d1c-33e512941c2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:33.369157 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e858eb-5c4b-4782-9d1c-33e512941c2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:33.370591 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e858eb-5c4b-4782-9d1c-33e512941c2a None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:33.388188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-b0e858eb-5c4b-4782-9d1c-33e512941c2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:33.388188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:33.388188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:33.388188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:33.388188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:33.388188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:33.388188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:33.388188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:33.388188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:33.388188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:33.388188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:33.388188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:33.388188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:33.388188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:33.388188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:33.389210 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:33.389210 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:33.389210 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:33.389210 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:33.389210 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:33.389210 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:33.389210 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:33.389210 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:33.389210 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:33.389210 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:33.389210 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:33.389210 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:33.389210 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:33.389210 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:33.389210 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:33.389210 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:33.389210 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:33.390211 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:33.390211 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:33.390211 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:33.390211 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:33.390211 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 434/861] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:33.395009 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdc529a-33b7-47fe-a3b7-7ea2f8d0e840 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:33.503580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdc529a-33b7-47fe-a3b7-7ea2f8d0e840 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:33.541651 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dfbef6-1540-47c2-b38a-f2d251916929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:33.542388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfbef6-1540-47c2-b38a-f2d251916929 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:33.542705 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfbef6-1540-47c2-b38a-f2d251916929 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:33.542984 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfbef6-1540-47c2-b38a-f2d251916929 None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:33.543735 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dfbef6-1540-47c2-b38a-f2d251916929 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:33.544045 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dfbef6-1540-47c2-b38a-f2d251916929 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:33.545534 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dfbef6-1540-47c2-b38a-f2d251916929 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:33.559302 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 428/862] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:33.564649 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e858eb-5c4b-4782-9d1c-33e512941c2a None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:33.668866 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e858eb-5c4b-4782-9d1c-33e512941c2a None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:33.706017 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c7cca-1ee4-494b-a4b2-9e1ae135ee40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:33.707725 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c7cca-1ee4-494b-a4b2-9e1ae135ee40 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:33.708644 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c7cca-1ee4-494b-a4b2-9e1ae135ee40 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:33.708943 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c7cca-1ee4-494b-a4b2-9e1ae135ee40 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:33.717904 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57c7cca-1ee4-494b-a4b2-9e1ae135ee40 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:33.719635 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57c7cca-1ee4-494b-a4b2-9e1ae135ee40 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:33.737685 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-b57c7cca-1ee4-494b-a4b2-9e1ae135ee40 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 06 14:45:33.737685 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:33.737685 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:33.737685 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:33.737685 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:33.737685 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:33.737685 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:33.737685 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:33.737685 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:33.737685 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:33.737685 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:33.737685 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:33.737685 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 06 14:45:33.737685 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_user(user_id) May 06 14:45:33.737685 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 06 14:45:33.737685 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 06 14:45:33.738699 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:33.738699 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:33.738699 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 06 14:45:33.738699 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:33.738699 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 06 14:45:33.738699 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:33.738699 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:33.738699 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:33.738699 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:33.738699 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:33.738699 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:33.738699 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:33.738699 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:33.738699 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:33.738699 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:33.738699 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:33.738699 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:33.738699 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:33.740516 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:33.740516 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:33.740516 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 06 14:45:33.740516 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 06 14:45:33.740516 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 06 14:45:33.740516 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 06 14:45:33.740516 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 06 14:45:33.740516 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 06 14:45:33.740516 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 06 14:45:33.740516 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:33.740516 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 435/863] 10.222.0.23 () {56 vars in 1117 bytes} [Fri May 6 14:45:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:33.747319 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dfbef6-1540-47c2-b38a-f2d251916929 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:33.854898 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dfbef6-1540-47c2-b38a-f2d251916929 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:33.889381 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b59cb6-5663-4c8b-b022-bdcb5e5c962d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:33.890074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b59cb6-5663-4c8b-b022-bdcb5e5c962d None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:33.890377 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b59cb6-5663-4c8b-b022-bdcb5e5c962d None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:33.890658 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b59cb6-5663-4c8b-b022-bdcb5e5c962d None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:33.891531 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b59cb6-5663-4c8b-b022-bdcb5e5c962d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:33.891842 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b59cb6-5663-4c8b-b022-bdcb5e5c962d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:33.893415 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b59cb6-5663-4c8b-b022-bdcb5e5c962d None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:33.914450 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 429/864] 10.222.0.23 () {56 vars in 1149 bytes} [Fri May 6 14:45:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:33.922062 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c7cca-1ee4-494b-a4b2-9e1ae135ee40 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:34.033251 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c7cca-1ee4-494b-a4b2-9e1ae135ee40 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:34.068106 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c799d98d-5a3e-478b-ab4f-aab44b9414fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:34.069154 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c799d98d-5a3e-478b-ab4f-aab44b9414fe None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:34.069675 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c799d98d-5a3e-478b-ab4f-aab44b9414fe None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:34.070156 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c799d98d-5a3e-478b-ab4f-aab44b9414fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:34.139610 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c799d98d-5a3e-478b-ab4f-aab44b9414fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:34.141442 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c799d98d-5a3e-478b-ab4f-aab44b9414fe None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:34.251340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 436/865] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:45:33 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 06 14:45:34.257679 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b59cb6-5663-4c8b-b022-bdcb5e5c962d None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:34.366262 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b59cb6-5663-4c8b-b022-bdcb5e5c962d None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:34.400405 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa705c07-77b5-455d-972f-68d2b8bc3923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:34.401171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa705c07-77b5-455d-972f-68d2b8bc3923 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:34.401438 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa705c07-77b5-455d-972f-68d2b8bc3923 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:34.401715 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa705c07-77b5-455d-972f-68d2b8bc3923 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:34.403295 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddace017-be7f-49be-9266-45729abf2f39 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:34.403799 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddace017-be7f-49be-9266-45729abf2f39 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:34.404287 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddace017-be7f-49be-9266-45729abf2f39 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:34.406388 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 437/866] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:45:34.412133 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76eeee7-8f63-4c78-9103-c06a7042d56e None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:34.412746 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76eeee7-8f63-4c78-9103-c06a7042d56e None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:34.412938 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa705c07-77b5-455d-972f-68d2b8bc3923 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:34.413413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76eeee7-8f63-4c78-9103-c06a7042d56e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:34.414371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa705c07-77b5-455d-972f-68d2b8bc3923 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:34.430965 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-f76eeee7-8f63-4c78-9103-c06a7042d56e None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:34.432470 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f76eeee7-8f63-4c78-9103-c06a7042d56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:34.433461 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-aa705c07-77b5-455d-972f-68d2b8bc3923 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:34.433461 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:34.433461 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:34.433461 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:34.433461 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:34.433461 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:34.433461 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:34.433461 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:34.433461 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:34.433461 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:34.433461 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:34.433461 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:34.433461 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:34.433461 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:34.433461 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:34.434344 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:34.434344 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:34.434344 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:34.434344 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:34.434344 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:34.434344 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:34.434344 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:34.434344 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:34.434344 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:34.434344 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:34.434344 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:34.434344 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:34.434344 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:34.434344 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:34.434344 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:34.434344 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:34.434344 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:34.435262 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:34.435262 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:34.435262 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:34.435262 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:34.435262 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 430/867] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:34.441493 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa705c07-77b5-455d-972f-68d2b8bc3923 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:34.508122 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 438/868] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:34 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 06 14:45:34.531288 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76eeee7-8f63-4c78-9103-c06a7042d56e None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:34.560382 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa705c07-77b5-455d-972f-68d2b8bc3923 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:34.605054 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573a8ac2-07d2-4fb4-8b8d-c55f78892c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:34.605725 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a8ac2-07d2-4fb4-8b8d-c55f78892c67 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:34.606012 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a8ac2-07d2-4fb4-8b8d-c55f78892c67 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:34.606292 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a8ac2-07d2-4fb4-8b8d-c55f78892c67 None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:34.607013 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573a8ac2-07d2-4fb4-8b8d-c55f78892c67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:34.607321 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573a8ac2-07d2-4fb4-8b8d-c55f78892c67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:34.608713 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573a8ac2-07d2-4fb4-8b8d-c55f78892c67 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:34.620581 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 431/869] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:34.626239 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573a8ac2-07d2-4fb4-8b8d-c55f78892c67 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:34.634090 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76eeee7-8f63-4c78-9103-c06a7042d56e None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:34.677264 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716d17bc-6d17-4281-b4ef-23e6ef82e736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:34.677676 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d17bc-6d17-4281-b4ef-23e6ef82e736 None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:34.677963 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d17bc-6d17-4281-b4ef-23e6ef82e736 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:34.678237 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d17bc-6d17-4281-b4ef-23e6ef82e736 None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:34.678963 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716d17bc-6d17-4281-b4ef-23e6ef82e736 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:34.680430 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716d17bc-6d17-4281-b4ef-23e6ef82e736 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:34.696711 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 439/870] 10.222.0.23 () {60 vars in 1141 bytes} [Fri May 6 14:45:34 2022] POST /identity/v3/services => generated 251 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 14:45:34.764302 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573a8ac2-07d2-4fb4-8b8d-c55f78892c67 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:34.811015 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f223a-0cb9-46eb-b9b6-79f6b111210e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:34.811884 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f223a-0cb9-46eb-b9b6-79f6b111210e None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:34.812172 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f223a-0cb9-46eb-b9b6-79f6b111210e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:34.812450 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f223a-0cb9-46eb-b9b6-79f6b111210e None admin] PATH_INFO: `/v3/projects/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:34.820829 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61f223a-0cb9-46eb-b9b6-79f6b111210e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:34.822349 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61f223a-0cb9-46eb-b9b6-79f6b111210e None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:34.838871 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-f61f223a-0cb9-46eb-b9b6-79f6b111210e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:34.838871 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:34.838871 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:34.838871 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:34.838871 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:34.838871 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:34.838871 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:34.838871 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:34.838871 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:34.838871 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:34.838871 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:34.838871 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:34.838871 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 06 14:45:34.838871 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(project_id) May 06 14:45:34.838871 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 06 14:45:34.841045 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 06 14:45:34.841045 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:34.841045 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:34.841045 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:34.841045 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:34.841045 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:34.841045 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:34.841045 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:34.841045 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:34.841045 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:34.841045 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:34.841045 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:34.841045 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:34.841045 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:34.841045 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:34.841045 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:34.841045 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:34.849688 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 06 14:45:34.849688 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 06 14:45:34.849688 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 06 14:45:34.849688 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 06 14:45:34.849688 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 06 14:45:34.849688 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 06 14:45:34.849688 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:34.849688 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:34.849688 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 432/871] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:45:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:45:34.850870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716d17bc-6d17-4281-b4ef-23e6ef82e736 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:34.967455 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716d17bc-6d17-4281-b4ef-23e6ef82e736 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:35.001617 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf94c79-161f-42fa-8ee0-c6bd754b9de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:35.002285 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf94c79-161f-42fa-8ee0-c6bd754b9de5 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:35.002572 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf94c79-161f-42fa-8ee0-c6bd754b9de5 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:35.002856 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf94c79-161f-42fa-8ee0-c6bd754b9de5 None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:35.003640 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf94c79-161f-42fa-8ee0-c6bd754b9de5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:35.003950 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf94c79-161f-42fa-8ee0-c6bd754b9de5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:35.006480 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf94c79-161f-42fa-8ee0-c6bd754b9de5 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:35.019390 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 440/872] 10.222.0.23 () {56 vars in 1148 bytes} [Fri May 6 14:45:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:35.026488 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f223a-0cb9-46eb-b9b6-79f6b111210e None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:35.139024 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f223a-0cb9-46eb-b9b6-79f6b111210e None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:35.188895 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5477165a-59c5-4df4-a6e6-ebf639f544d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:35.190010 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477165a-59c5-4df4-a6e6-ebf639f544d7 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:35.190500 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477165a-59c5-4df4-a6e6-ebf639f544d7 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:35.190984 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477165a-59c5-4df4-a6e6-ebf639f544d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:35.192643 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5477165a-59c5-4df4-a6e6-ebf639f544d7 None admin] RBAC: Adding query filter params (role.id=f5641817ee56438496afc6135d37a4c0, scope.project.id=d5398feee6eb490cb49bb261448a1a9b, user.id=a7dc1bc53c59423e934f7831ea022c73) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:35.193167 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5477165a-59c5-4df4-a6e6-ebf639f544d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:35.194822 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5477165a-59c5-4df4-a6e6-ebf639f544d7 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:35.201732 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 433/873] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:45:35 2022] GET /identity/v3/role_assignments?user.id=a7dc1bc53c59423e934f7831ea022c73&scope.project.id=d5398feee6eb490cb49bb261448a1a9b&role.id=f5641817ee56438496afc6135d37a4c0 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:35.230963 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70979a01-5c4b-4532-806c-3a946353dd96 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:35.231157 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70979a01-5c4b-4532-806c-3a946353dd96 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:35.231440 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70979a01-5c4b-4532-806c-3a946353dd96 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:35.233231 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 441/874] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:45:35.240038 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f6f0f0-61f1-4f81-a03f-326f3c17e5a3 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:35.240744 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f6f0f0-61f1-4f81-a03f-326f3c17e5a3 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:35.241228 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f6f0f0-61f1-4f81-a03f-326f3c17e5a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:35.260465 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-d5f6f0f0-61f1-4f81-a03f-326f3c17e5a3 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:35.262268 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d5f6f0f0-61f1-4f81-a03f-326f3c17e5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:35.343716 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 434/875] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:35 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:45:35.366112 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70979a01-5c4b-4532-806c-3a946353dd96 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:35.463956 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8de959-01ae-4520-8764-3f7456978cd4 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:35.464498 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8de959-01ae-4520-8764-3f7456978cd4 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:35.464787 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8de959-01ae-4520-8764-3f7456978cd4 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:35.466247 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 435/876] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:45:35.471434 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba705373-c56e-4146-8ae6-d2581f888811 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:35.471826 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba705373-c56e-4146-8ae6-d2581f888811 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:35.472204 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba705373-c56e-4146-8ae6-d2581f888811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:35.479578 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70979a01-5c4b-4532-806c-3a946353dd96 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:35.493021 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-ba705373-c56e-4146-8ae6-d2581f888811 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:35.494567 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ba705373-c56e-4146-8ae6-d2581f888811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:35.528101 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05592c7-a075-403d-a6a8-872e1e438de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:35.528960 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05592c7-a075-403d-a6a8-872e1e438de2 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:35.529279 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05592c7-a075-403d-a6a8-872e1e438de2 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:35.529561 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05592c7-a075-403d-a6a8-872e1e438de2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:35.530293 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05592c7-a075-403d-a6a8-872e1e438de2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:35.531999 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05592c7-a075-403d-a6a8-872e1e438de2 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:35.548904 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-d05592c7-a075-403d-a6a8-872e1e438de2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 06 14:45:35.548904 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:35.548904 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 06 14:45:35.548904 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 06 14:45:35.548904 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 06 14:45:35.548904 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 06 14:45:35.548904 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 06 14:45:35.548904 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:35.548904 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 06 14:45:35.548904 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:35.548904 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 06 14:45:35.548904 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:35.548904 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:35.548904 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:35.548904 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:35.548904 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:35.548904 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:35.550096 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:35.550096 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:35.550096 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:35.550096 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:35.550096 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:35.550096 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:35.550096 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 06 14:45:35.550096 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(service_id) May 06 14:45:35.550096 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 06 14:45:35.550096 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 06 14:45:35.550096 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:35.550096 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:35.550096 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:35.550096 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:35.550096 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:35.550096 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:35.550975 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:35.550975 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:35.550975 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:35.550975 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:35.550975 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:35.550975 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:35.550975 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:35.550975 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:35.550975 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:35.550975 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:35.550975 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 06 14:45:35.550975 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:35.550975 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 06 14:45:35.550975 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:35.550975 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 442/877] 10.222.0.23 () {56 vars in 1126 bytes} [Fri May 6 14:45:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:35.555951 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05592c7-a075-403d-a6a8-872e1e438de2 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:35.594027 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 436/878] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:35 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 06 14:45:35.619302 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba705373-c56e-4146-8ae6-d2581f888811 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:35.667259 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05592c7-a075-403d-a6a8-872e1e438de2 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:35.702969 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d38e0-4664-4d21-9ca9-adbc71f3de03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:35.703916 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d38e0-4664-4d21-9ca9-adbc71f3de03 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:35.704368 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d38e0-4664-4d21-9ca9-adbc71f3de03 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:35.704816 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d38e0-4664-4d21-9ca9-adbc71f3de03 None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:35.705665 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d38e0-4664-4d21-9ca9-adbc71f3de03 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:35.706118 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d38e0-4664-4d21-9ca9-adbc71f3de03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:35.708269 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d38e0-4664-4d21-9ca9-adbc71f3de03 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:35.714124 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 443/879] 10.222.0.23 () {56 vars in 1122 bytes} [Fri May 6 14:45:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:35.719909 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d38e0-4664-4d21-9ca9-adbc71f3de03 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:35.723144 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba705373-c56e-4146-8ae6-d2581f888811 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:35.759877 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede422a-7bd9-43ad-bccc-27049bc762d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:35.760629 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede422a-7bd9-43ad-bccc-27049bc762d8 None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:35.761017 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede422a-7bd9-43ad-bccc-27049bc762d8 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:35.761394 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede422a-7bd9-43ad-bccc-27049bc762d8 None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:35.762284 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ede422a-7bd9-43ad-bccc-27049bc762d8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:35.763777 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ede422a-7bd9-43ad-bccc-27049bc762d8 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:35.784680 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 437/880] 10.222.0.23 () {60 vars in 1141 bytes} [Fri May 6 14:45:35 2022] POST /identity/v3/services => generated 269 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 14:45:35.844135 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d38e0-4664-4d21-9ca9-adbc71f3de03 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:35.889313 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95063c96-f08c-4ba0-adff-67f5177d00e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:35.890318 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95063c96-f08c-4ba0-adff-67f5177d00e9 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:35.890775 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95063c96-f08c-4ba0-adff-67f5177d00e9 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:35.891198 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95063c96-f08c-4ba0-adff-67f5177d00e9 None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:35.892078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95063c96-f08c-4ba0-adff-67f5177d00e9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:35.892535 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95063c96-f08c-4ba0-adff-67f5177d00e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:35.894091 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95063c96-f08c-4ba0-adff-67f5177d00e9 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:35.900563 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 444/881] 10.222.0.23 () {56 vars in 1122 bytes} [Fri May 6 14:45:35 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:37.093228 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e9989-edb5-4d08-bd47-44377516dade None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:37.093762 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e9989-edb5-4d08-bd47-44377516dade None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:37.093762 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e9989-edb5-4d08-bd47-44377516dade None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:37.095369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 438/882] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:45:37.101392 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c41dd-605c-461d-981c-67ae82413fb6 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:37.101853 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c41dd-605c-461d-981c-67ae82413fb6 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:37.101954 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c41dd-605c-461d-981c-67ae82413fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:37.119978 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-644c41dd-605c-461d-981c-67ae82413fb6 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:37.121560 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-644c41dd-605c-461d-981c-67ae82413fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:37.194564 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 445/883] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:37 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:45:37.217722 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e9989-edb5-4d08-bd47-44377516dade None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:37.324355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e9989-edb5-4d08-bd47-44377516dade None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:37.359818 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e866efed-d823-4805-a01b-a2af8473a876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:37.360527 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866efed-d823-4805-a01b-a2af8473a876 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:37.360831 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866efed-d823-4805-a01b-a2af8473a876 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:37.361104 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866efed-d823-4805-a01b-a2af8473a876 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:37.361741 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e866efed-d823-4805-a01b-a2af8473a876 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:37.363087 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e866efed-d823-4805-a01b-a2af8473a876 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:37.375731 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-e866efed-d823-4805-a01b-a2af8473a876 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 06 14:45:37.375731 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:37.375731 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 06 14:45:37.375731 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 06 14:45:37.375731 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 06 14:45:37.375731 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 06 14:45:37.375731 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 06 14:45:37.375731 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:37.375731 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 06 14:45:37.375731 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:37.375731 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 06 14:45:37.375731 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:37.375731 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:37.375731 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:37.375731 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:37.375731 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:37.375731 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:37.376759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:37.376759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:37.376759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:37.376759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:37.376759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:37.376759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:37.376759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 06 14:45:37.376759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(service_id) May 06 14:45:37.376759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 06 14:45:37.376759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 06 14:45:37.376759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:37.376759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:37.376759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:37.376759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:37.376759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:37.376759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:37.377616 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:37.377616 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:37.377616 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:37.377616 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:37.377616 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:37.377616 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:37.377616 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:37.377616 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:37.377616 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:37.377616 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:37.377616 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 06 14:45:37.377616 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:37.377616 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 06 14:45:37.377616 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:37.380701 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 439/884] 10.222.0.23 () {56 vars in 1114 bytes} [Fri May 6 14:45:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:37.383437 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644c41dd-605c-461d-981c-67ae82413fb6 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:37.497930 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644c41dd-605c-461d-981c-67ae82413fb6 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:37.534419 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a337a949-f745-4766-8532-8df3b30ceaa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:37.535535 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a337a949-f745-4766-8532-8df3b30ceaa6 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:37.536107 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a337a949-f745-4766-8532-8df3b30ceaa6 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:37.536628 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a337a949-f745-4766-8532-8df3b30ceaa6 None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:37.537559 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a337a949-f745-4766-8532-8df3b30ceaa6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:37.538072 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a337a949-f745-4766-8532-8df3b30ceaa6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:37.539721 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a337a949-f745-4766-8532-8df3b30ceaa6 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:37.545334 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 446/885] 10.222.0.23 () {56 vars in 1114 bytes} [Fri May 6 14:45:37 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:37.550805 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e866efed-d823-4805-a01b-a2af8473a876 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:37.639594 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b71b9d-69de-41a2-be7f-542378cd14a2 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:37.639887 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b71b9d-69de-41a2-be7f-542378cd14a2 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:37.640086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b71b9d-69de-41a2-be7f-542378cd14a2 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:37.641725 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 447/886] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:45:37.647041 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de72d788-99e5-49d1-80fd-3a0b81989af6 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:37.647328 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de72d788-99e5-49d1-80fd-3a0b81989af6 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:37.647607 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de72d788-99e5-49d1-80fd-3a0b81989af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:37.658548 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e866efed-d823-4805-a01b-a2af8473a876 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:37.665055 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-de72d788-99e5-49d1-80fd-3a0b81989af6 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:37.666494 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-de72d788-99e5-49d1-80fd-3a0b81989af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:37.694850 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cf3c80-f273-4526-8df7-f4d060baafc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:37.695566 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf3c80-f273-4526-8df7-f4d060baafc8 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:37.695875 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf3c80-f273-4526-8df7-f4d060baafc8 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:37.696194 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf3c80-f273-4526-8df7-f4d060baafc8 None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:37.697049 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cf3c80-f273-4526-8df7-f4d060baafc8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:37.697362 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cf3c80-f273-4526-8df7-f4d060baafc8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:37.698855 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cf3c80-f273-4526-8df7-f4d060baafc8 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:37.709375 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 440/887] 10.222.0.23 () {56 vars in 1114 bytes} [Fri May 6 14:45:37 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:37.710871 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cf3c80-f273-4526-8df7-f4d060baafc8 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:37.744053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 448/888] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:37 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 06 14:45:37.767526 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de72d788-99e5-49d1-80fd-3a0b81989af6 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:37.820385 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cf3c80-f273-4526-8df7-f4d060baafc8 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:37.854265 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95c504-883c-49b7-b32a-4ab078ce824d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:37.854922 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95c504-883c-49b7-b32a-4ab078ce824d None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:37.855199 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95c504-883c-49b7-b32a-4ab078ce824d None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:37.855468 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95c504-883c-49b7-b32a-4ab078ce824d None admin] PATH_INFO: `/v3/endpoints` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:37.856247 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d95c504-883c-49b7-b32a-4ab078ce824d None admin] RBAC: Adding query filter params (interface=public, service_id=14d6eb82e0c64571b328b6d072dee6ce, region_id=RegionOne) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:37.856557 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d95c504-883c-49b7-b32a-4ab078ce824d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:37.857934 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d95c504-883c-49b7-b32a-4ab078ce824d None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:37.868223 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 441/889] 10.222.0.23 () {56 vars in 1251 bytes} [Fri May 6 14:45:37 2022] GET /identity/v3/endpoints?service_id=14d6eb82e0c64571b328b6d072dee6ce&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:37.869427 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de72d788-99e5-49d1-80fd-3a0b81989af6 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:37.869635 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95c504-883c-49b7-b32a-4ab078ce824d None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:37.913613 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75762ec-2132-4f1d-807b-9400d5be64fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:37.915515 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75762ec-2132-4f1d-807b-9400d5be64fa None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:37.916067 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75762ec-2132-4f1d-807b-9400d5be64fa None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:37.917256 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75762ec-2132-4f1d-807b-9400d5be64fa None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:37.933945 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75762ec-2132-4f1d-807b-9400d5be64fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:45:37.946340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-d75762ec-2132-4f1d-807b-9400d5be64fa None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 06 14:45:37.946340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:37.946340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:37.946340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:37.946340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:37.946340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:37.946340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:37.946340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:37.946340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:37.946340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:37.946340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:37.946340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:37.946340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:45:37.946340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:45:37.946340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:45:37.946340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:45:37.947741 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:45:37.947741 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:45:37.947741 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:45:37.947741 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:45:37.947741 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:37.947741 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:37.947741 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:37.947741 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:37.947741 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:37.947741 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:37.947741 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:37.947741 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:37.947741 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:37.947741 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:37.947741 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:37.947741 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:37.949018 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:37.949018 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:37.949018 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:37.949018 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:37.949018 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:45:37.949018 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:45:37.949018 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:45:37.949018 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:45:37.949018 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:45:37.949018 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:45:37.949018 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 06 14:45:37.949018 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:37.962706 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 449/890] 10.222.0.23 () {56 vars in 1120 bytes} [Fri May 6 14:45:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:45:37.965809 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75762ec-2132-4f1d-807b-9400d5be64fa None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:38.013220 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95c504-883c-49b7-b32a-4ab078ce824d None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:38.057611 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030cc53-edaa-4459-b406-20dcf5ec055a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:38.058740 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030cc53-edaa-4459-b406-20dcf5ec055a None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:38.059260 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030cc53-edaa-4459-b406-20dcf5ec055a None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:38.059750 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030cc53-edaa-4459-b406-20dcf5ec055a None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:38.060706 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a030cc53-edaa-4459-b406-20dcf5ec055a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:38.063041 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a030cc53-edaa-4459-b406-20dcf5ec055a None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:38.081325 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 442/891] 10.222.0.23 () {56 vars in 1087 bytes} [Fri May 6 14:45:37 2022] GET /identity/v3/services => generated 1268 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 06 14:45:38.095353 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75762ec-2132-4f1d-807b-9400d5be64fa None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:38.118646 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5864a8-59eb-4c3f-9015-a0cabb1c0bfa None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:38.119212 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5864a8-59eb-4c3f-9015-a0cabb1c0bfa None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:38.119212 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5864a8-59eb-4c3f-9015-a0cabb1c0bfa None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:38.120880 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 443/892] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:45:38.130507 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bad367-20de-4a93-a455-75c5686d857c None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:38.130823 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bad367-20de-4a93-a455-75c5686d857c None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:38.131067 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bad367-20de-4a93-a455-75c5686d857c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:38.141295 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e44a20-8c56-457e-8369-8c99208b7d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:38.142740 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e44a20-8c56-457e-8369-8c99208b7d92 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:38.143231 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e44a20-8c56-457e-8369-8c99208b7d92 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:38.143723 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e44a20-8c56-457e-8369-8c99208b7d92 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:38.144674 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e44a20-8c56-457e-8369-8c99208b7d92 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:38.145837 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e44a20-8c56-457e-8369-8c99208b7d92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:38.147424 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e44a20-8c56-457e-8369-8c99208b7d92 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:38.151897 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-09bad367-20de-4a93-a455-75c5686d857c None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:38.153358 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-09bad367-20de-4a93-a455-75c5686d857c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:38.158179 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 450/893] 10.222.0.23 () {56 vars in 1115 bytes} [Fri May 6 14:45:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:38.164276 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e44a20-8c56-457e-8369-8c99208b7d92 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:38.241306 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 444/894] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:38 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:45:38.263515 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bad367-20de-4a93-a455-75c5686d857c None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:38.288973 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e44a20-8c56-457e-8369-8c99208b7d92 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:38.327777 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38285c-6961-4ec0-a1bc-a21ce2a9ebd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:38.328577 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38285c-6961-4ec0-a1bc-a21ce2a9ebd5 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:38.328884 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38285c-6961-4ec0-a1bc-a21ce2a9ebd5 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:38.329376 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38285c-6961-4ec0-a1bc-a21ce2a9ebd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:38.340672 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f38285c-6961-4ec0-a1bc-a21ce2a9ebd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:38.342164 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f38285c-6961-4ec0-a1bc-a21ce2a9ebd5 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:38.362659 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-4f38285c-6961-4ec0-a1bc-a21ce2a9ebd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:38.362659 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:38.362659 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:38.362659 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:38.362659 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:38.362659 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:38.362659 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:38.362659 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:38.362659 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:38.362659 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:38.362659 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:38.362659 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:38.362659 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:38.362659 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:38.362659 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:38.363639 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:38.363639 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:38.363639 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:38.363639 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:38.363639 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:38.363639 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:38.363639 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:38.363639 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:38.363639 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:38.363639 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:38.363639 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:38.363639 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:38.363639 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:38.363639 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:38.363639 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:38.363639 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:38.363639 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:38.364564 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:38.364564 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:38.364564 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:38.364564 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:38.364564 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 451/895] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:38.365383 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802b1b1-bf93-48a2-b311-1d3fc0058a53 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:38.365485 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802b1b1-bf93-48a2-b311-1d3fc0058a53 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:38.365769 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802b1b1-bf93-48a2-b311-1d3fc0058a53 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:38.370987 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 452/896] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:45:38.370987 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4802b1b1-bf93-48a2-b311-1d3fc0058a53 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:38.378757 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bad367-20de-4a93-a455-75c5686d857c None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:38.419696 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a4e32a-a61d-4bbb-bda9-ea23e50d244c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:38.420419 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a4e32a-a61d-4bbb-bda9-ea23e50d244c None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:38.420738 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a4e32a-a61d-4bbb-bda9-ea23e50d244c None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:38.421022 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a4e32a-a61d-4bbb-bda9-ea23e50d244c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:38.421670 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a4e32a-a61d-4bbb-bda9-ea23e50d244c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:38.423033 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a4e32a-a61d-4bbb-bda9-ea23e50d244c None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:38.435376 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-91a4e32a-a61d-4bbb-bda9-ea23e50d244c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 06 14:45:38.435376 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:38.435376 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 06 14:45:38.435376 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 06 14:45:38.435376 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 06 14:45:38.435376 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 06 14:45:38.435376 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 06 14:45:38.435376 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:38.435376 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 06 14:45:38.435376 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:38.435376 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 06 14:45:38.435376 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:38.435376 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:38.435376 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:38.435376 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:38.435376 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:38.436349 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:38.436349 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:38.436349 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:38.436349 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:38.436349 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:38.436349 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:38.436349 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:38.436349 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 06 14:45:38.436349 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(service_id) May 06 14:45:38.436349 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 06 14:45:38.436349 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 06 14:45:38.436349 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:38.436349 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:38.436349 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:38.436349 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:38.436349 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:38.437253 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:38.437253 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:38.437253 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:38.437253 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:38.437253 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:38.437253 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:38.437253 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:38.437253 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:38.437253 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:38.437253 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:38.437253 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:38.437253 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 06 14:45:38.437253 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:38.437253 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 06 14:45:38.437253 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:38.437253 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 445/897] 10.222.0.23 () {56 vars in 1132 bytes} [Fri May 6 14:45:38 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:45:38.438632 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b54be4-c67b-462f-a39c-06063f7e65f2 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:38.438632 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b54be4-c67b-462f-a39c-06063f7e65f2 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:38.438788 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b54be4-c67b-462f-a39c-06063f7e65f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:38.455229 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-63b54be4-c67b-462f-a39c-06063f7e65f2 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:38.456413 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-63b54be4-c67b-462f-a39c-06063f7e65f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:38.483183 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4802b1b1-bf93-48a2-b311-1d3fc0058a53 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:38.517212 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3d0f1b-45c2-4f37-8489-2a9c5f5027ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:38.517879 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d0f1b-45c2-4f37-8489-2a9c5f5027ce None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:38.518168 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d0f1b-45c2-4f37-8489-2a9c5f5027ce None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:38.518445 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d0f1b-45c2-4f37-8489-2a9c5f5027ce None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:38.519165 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3d0f1b-45c2-4f37-8489-2a9c5f5027ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:38.519471 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3d0f1b-45c2-4f37-8489-2a9c5f5027ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:38.521520 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3d0f1b-45c2-4f37-8489-2a9c5f5027ce None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:38.530146 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 446/898] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:38 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:45:38.531452 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b54be4-c67b-462f-a39c-06063f7e65f2 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:38.544760 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 453/899] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:38.557611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3d0f1b-45c2-4f37-8489-2a9c5f5027ce None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:38.637141 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b54be4-c67b-462f-a39c-06063f7e65f2 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:38.657988 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3d0f1b-45c2-4f37-8489-2a9c5f5027ce None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:38.678616 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974a7cf3-c5e5-49a9-a2cd-c1df2e77aec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:38.679221 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a7cf3-c5e5-49a9-a2cd-c1df2e77aec5 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:38.679503 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a7cf3-c5e5-49a9-a2cd-c1df2e77aec5 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:38.679778 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a7cf3-c5e5-49a9-a2cd-c1df2e77aec5 None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:38.680487 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974a7cf3-c5e5-49a9-a2cd-c1df2e77aec5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:38.680824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974a7cf3-c5e5-49a9-a2cd-c1df2e77aec5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:38.682197 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974a7cf3-c5e5-49a9-a2cd-c1df2e77aec5 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:38.686910 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 447/900] 10.222.0.23 () {56 vars in 1126 bytes} [Fri May 6 14:45:38 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:38.687679 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974a7cf3-c5e5-49a9-a2cd-c1df2e77aec5 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:38.698381 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bf9530-f276-426b-8570-d661f7f2b46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:38.699231 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf9530-f276-426b-8570-d661f7f2b46b None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:38.699514 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf9530-f276-426b-8570-d661f7f2b46b None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:38.699815 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf9530-f276-426b-8570-d661f7f2b46b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:38.707145 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bf9530-f276-426b-8570-d661f7f2b46b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:38.709415 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bf9530-f276-426b-8570-d661f7f2b46b None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:38.725186 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-65bf9530-f276-426b-8570-d661f7f2b46b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 06 14:45:38.725186 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:38.725186 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:38.725186 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:38.725186 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:38.725186 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:38.725186 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:38.725186 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:38.725186 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:38.725186 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:38.725186 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:38.725186 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:38.725186 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 06 14:45:38.725186 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_user(user_id) May 06 14:45:38.725186 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 06 14:45:38.725186 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 06 14:45:38.726174 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:38.726174 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:38.726174 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 06 14:45:38.726174 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:38.726174 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 06 14:45:38.726174 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:38.726174 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:38.726174 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:38.726174 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:38.726174 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:38.726174 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:38.726174 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:38.726174 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:38.726174 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:38.726174 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:38.726174 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:38.726174 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:38.726174 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:38.732314 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:38.732314 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:38.732314 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 06 14:45:38.732314 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 06 14:45:38.732314 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 06 14:45:38.732314 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 06 14:45:38.732314 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 06 14:45:38.732314 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 06 14:45:38.732314 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 06 14:45:38.732314 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:38.732314 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 454/901] 10.222.0.23 () {56 vars in 1117 bytes} [Fri May 6 14:45:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:38.732314 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bf9530-f276-426b-8570-d661f7f2b46b None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:38.790994 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974a7cf3-c5e5-49a9-a2cd-c1df2e77aec5 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:38.824711 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3be81-1e6f-4051-bff0-10c13a5e2ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:38.825383 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3be81-1e6f-4051-bff0-10c13a5e2ddb None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:38.825665 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3be81-1e6f-4051-bff0-10c13a5e2ddb None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:38.825942 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3be81-1e6f-4051-bff0-10c13a5e2ddb None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:38.826658 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e3be81-1e6f-4051-bff0-10c13a5e2ddb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:38.828016 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e3be81-1e6f-4051-bff0-10c13a5e2ddb None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:38.840285 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bf9530-f276-426b-8570-d661f7f2b46b None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:38.843122 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 448/902] 10.222.0.23 () {60 vars in 1141 bytes} [Fri May 6 14:45:38 2022] POST /identity/v3/services => generated 241 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:45:38.844081 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3be81-1e6f-4051-bff0-10c13a5e2ddb None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:38.894882 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c04667-f5ea-45fb-bc82-421f618aefb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:38.896552 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c04667-f5ea-45fb-bc82-421f618aefb5 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:38.897018 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c04667-f5ea-45fb-bc82-421f618aefb5 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:38.897852 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c04667-f5ea-45fb-bc82-421f618aefb5 None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:38.899023 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c04667-f5ea-45fb-bc82-421f618aefb5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:38.899522 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c04667-f5ea-45fb-bc82-421f618aefb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:38.901317 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c04667-f5ea-45fb-bc82-421f618aefb5 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:38.909187 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 455/903] 10.222.0.23 () {56 vars in 1126 bytes} [Fri May 6 14:45:38 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:38.915497 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c04667-f5ea-45fb-bc82-421f618aefb5 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:38.988113 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3be81-1e6f-4051-bff0-10c13a5e2ddb None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:39.026740 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df080272-c628-4676-bf3a-8d434acaf65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:39.027822 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df080272-c628-4676-bf3a-8d434acaf65a None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:39.028307 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df080272-c628-4676-bf3a-8d434acaf65a None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:39.028830 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df080272-c628-4676-bf3a-8d434acaf65a None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:39.029874 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df080272-c628-4676-bf3a-8d434acaf65a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:39.030673 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df080272-c628-4676-bf3a-8d434acaf65a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:39.032316 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df080272-c628-4676-bf3a-8d434acaf65a None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:39.050007 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c04667-f5ea-45fb-bc82-421f618aefb5 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:39.055717 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 449/904] 10.222.0.23 () {56 vars in 1149 bytes} [Fri May 6 14:45:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:39.063716 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df080272-c628-4676-bf3a-8d434acaf65a None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:39.091621 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555d22ba-04b6-4083-91f3-b2d68dc4f0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:39.092289 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d22ba-04b6-4083-91f3-b2d68dc4f0a8 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:39.092575 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d22ba-04b6-4083-91f3-b2d68dc4f0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:39.092880 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d22ba-04b6-4083-91f3-b2d68dc4f0a8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:39.094038 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555d22ba-04b6-4083-91f3-b2d68dc4f0a8 None admin] RBAC: Adding query filter params (interface=public, service_id=7bd58f2a0437461bb79c3c8a849cb206, region_id=RegionOne) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:39.094038 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555d22ba-04b6-4083-91f3-b2d68dc4f0a8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:39.095343 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555d22ba-04b6-4083-91f3-b2d68dc4f0a8 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:39.103825 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 456/905] 10.222.0.23 () {56 vars in 1251 bytes} [Fri May 6 14:45:38 2022] GET /identity/v3/endpoints?service_id=7bd58f2a0437461bb79c3c8a849cb206&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:39.109610 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555d22ba-04b6-4083-91f3-b2d68dc4f0a8 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:39.187806 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df080272-c628-4676-bf3a-8d434acaf65a None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:39.214655 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555d22ba-04b6-4083-91f3-b2d68dc4f0a8 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:39.229637 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b0141e-a7fa-4194-944f-ffc0f65e3aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:39.230409 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b0141e-a7fa-4194-944f-ffc0f65e3aea None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:39.230715 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b0141e-a7fa-4194-944f-ffc0f65e3aea None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:39.231171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b0141e-a7fa-4194-944f-ffc0f65e3aea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:39.256814 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f73090-2557-4c5c-86ee-50d3a7c68e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:39.257568 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f73090-2557-4c5c-86ee-50d3a7c68e85 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:39.257678 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f73090-2557-4c5c-86ee-50d3a7c68e85 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:39.257954 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f73090-2557-4c5c-86ee-50d3a7c68e85 None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:39.258641 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f73090-2557-4c5c-86ee-50d3a7c68e85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:39.260191 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f73090-2557-4c5c-86ee-50d3a7c68e85 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:39.276958 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 457/906] 10.222.0.23 () {56 vars in 1087 bytes} [Fri May 6 14:45:39 2022] GET /identity/v3/services => generated 1497 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 06 14:45:39.316316 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b0141e-a7fa-4194-944f-ffc0f65e3aea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:39.318736 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b0141e-a7fa-4194-944f-ffc0f65e3aea None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:39.434495 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 450/907] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:45:39 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 06 14:45:39.440139 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f73090-2557-4c5c-86ee-50d3a7c68e85 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:39.549910 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f73090-2557-4c5c-86ee-50d3a7c68e85 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:39.584874 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0354cb-fa54-42f1-a6e3-e0634a2c6675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:39.585586 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0354cb-fa54-42f1-a6e3-e0634a2c6675 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:39.585871 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0354cb-fa54-42f1-a6e3-e0634a2c6675 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:39.586140 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0354cb-fa54-42f1-a6e3-e0634a2c6675 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:39.596575 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0354cb-fa54-42f1-a6e3-e0634a2c6675 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:39.598208 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0354cb-fa54-42f1-a6e3-e0634a2c6675 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:39.615520 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-6d0354cb-fa54-42f1-a6e3-e0634a2c6675 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:39.615520 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:39.615520 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:39.615520 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:39.615520 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:39.615520 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:39.615520 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:39.615520 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:39.615520 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:39.615520 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:39.615520 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:39.615520 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:39.615520 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:39.615520 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:39.615520 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:39.616453 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:39.616453 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:39.616453 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:39.616453 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:39.616453 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:39.616453 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:39.616453 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:39.616453 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:39.616453 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:39.616453 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:39.616453 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:39.616453 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:39.616453 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:39.616453 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:39.616453 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:39.616453 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:39.616453 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:39.617381 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:39.617381 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:39.617381 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:39.617381 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:39.617381 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 458/908] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:39.622361 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b0141e-a7fa-4194-944f-ffc0f65e3aea None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:39.722736 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b0141e-a7fa-4194-944f-ffc0f65e3aea None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:39.756784 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10712467-15e6-4c19-971f-adc4168dda97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:39.757454 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10712467-15e6-4c19-971f-adc4168dda97 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:39.757740 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10712467-15e6-4c19-971f-adc4168dda97 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:39.758018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10712467-15e6-4c19-971f-adc4168dda97 None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:39.758834 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10712467-15e6-4c19-971f-adc4168dda97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:39.759144 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10712467-15e6-4c19-971f-adc4168dda97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:39.760564 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10712467-15e6-4c19-971f-adc4168dda97 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:39.773026 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 451/909] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:39.778605 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0354cb-fa54-42f1-a6e3-e0634a2c6675 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:39.887853 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0354cb-fa54-42f1-a6e3-e0634a2c6675 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:39.922823 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935295f7-be63-4246-93c4-997d2f9b1d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:39.924003 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935295f7-be63-4246-93c4-997d2f9b1d1e None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:39.924495 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935295f7-be63-4246-93c4-997d2f9b1d1e None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:39.924991 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935295f7-be63-4246-93c4-997d2f9b1d1e None admin] PATH_INFO: `/v3/projects/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:39.932806 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935295f7-be63-4246-93c4-997d2f9b1d1e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:39.934491 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935295f7-be63-4246-93c4-997d2f9b1d1e None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:39.949413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-935295f7-be63-4246-93c4-997d2f9b1d1e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:39.949413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:39.949413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:39.949413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:39.949413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:39.949413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:39.949413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:39.949413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:39.949413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:39.949413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:39.949413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:39.949413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:39.949413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 06 14:45:39.949413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(project_id) May 06 14:45:39.949413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 06 14:45:39.950334 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 06 14:45:39.950334 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:39.950334 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:39.950334 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:39.950334 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:39.950334 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:39.950334 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:39.950334 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:39.950334 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:39.950334 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:39.950334 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:39.950334 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:39.950334 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:39.950334 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:39.950334 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:39.950334 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:39.950334 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:39.951217 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 06 14:45:39.951217 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 06 14:45:39.951217 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 06 14:45:39.951217 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 06 14:45:39.951217 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 06 14:45:39.951217 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 06 14:45:39.951217 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:39.951217 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:39.953727 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 459/910] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:45:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:39.960018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10712467-15e6-4c19-971f-adc4168dda97 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:40.064462 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10712467-15e6-4c19-971f-adc4168dda97 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:40.100271 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2339689e-3e78-41e4-81d3-ed2b07cdf042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:40.100988 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2339689e-3e78-41e4-81d3-ed2b07cdf042 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:40.101277 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2339689e-3e78-41e4-81d3-ed2b07cdf042 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:40.101557 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2339689e-3e78-41e4-81d3-ed2b07cdf042 None admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:40.102373 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2339689e-3e78-41e4-81d3-ed2b07cdf042 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:40.102686 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2339689e-3e78-41e4-81d3-ed2b07cdf042 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:40.104170 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2339689e-3e78-41e4-81d3-ed2b07cdf042 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:40.119525 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 452/911] 10.222.0.23 () {56 vars in 1148 bytes} [Fri May 6 14:45:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:40.125500 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935295f7-be63-4246-93c4-997d2f9b1d1e None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:40.238166 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935295f7-be63-4246-93c4-997d2f9b1d1e None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:40.278054 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f61cce3-8b72-46f9-9d9f-923edc4cd381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:40.279110 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f61cce3-8b72-46f9-9d9f-923edc4cd381 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:40.279621 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f61cce3-8b72-46f9-9d9f-923edc4cd381 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:40.280101 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f61cce3-8b72-46f9-9d9f-923edc4cd381 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:40.281468 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f61cce3-8b72-46f9-9d9f-923edc4cd381 None admin] RBAC: Adding query filter params (role.id=5213da7472024e87a41cecfaa37abce3, scope.project.id=d5398feee6eb490cb49bb261448a1a9b, user.id=a7dc1bc53c59423e934f7831ea022c73) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:40.282005 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f61cce3-8b72-46f9-9d9f-923edc4cd381 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:40.283694 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f61cce3-8b72-46f9-9d9f-923edc4cd381 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:40.291757 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 460/912] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:45:40 2022] GET /identity/v3/role_assignments?user.id=a7dc1bc53c59423e934f7831ea022c73&scope.project.id=d5398feee6eb490cb49bb261448a1a9b&role.id=5213da7472024e87a41cecfaa37abce3 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:40.465439 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2259b0e5-b1da-439c-8b05-6d53605566ab None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:40.465439 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2259b0e5-b1da-439c-8b05-6d53605566ab None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:40.465439 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2259b0e5-b1da-439c-8b05-6d53605566ab None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:40.466823 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 453/913] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:45:40.472520 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad6cf84-931b-4d82-9cfa-b68926aad215 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:40.473396 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad6cf84-931b-4d82-9cfa-b68926aad215 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:40.473657 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad6cf84-931b-4d82-9cfa-b68926aad215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:40.493289 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-9ad6cf84-931b-4d82-9cfa-b68926aad215 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:40.494763 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-9ad6cf84-931b-4d82-9cfa-b68926aad215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:40.572715 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 461/914] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:45:40.589380 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2259b0e5-b1da-439c-8b05-6d53605566ab None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:40.695382 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2259b0e5-b1da-439c-8b05-6d53605566ab None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:40.731470 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b640a27-6e47-4389-a2ea-95d5bbd54316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:40.732203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b640a27-6e47-4389-a2ea-95d5bbd54316 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:40.732487 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b640a27-6e47-4389-a2ea-95d5bbd54316 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:40.732793 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b640a27-6e47-4389-a2ea-95d5bbd54316 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:40.733443 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b640a27-6e47-4389-a2ea-95d5bbd54316 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:40.734802 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b640a27-6e47-4389-a2ea-95d5bbd54316 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:40.746905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-1b640a27-6e47-4389-a2ea-95d5bbd54316 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 06 14:45:40.746905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:40.746905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 06 14:45:40.746905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 06 14:45:40.746905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 06 14:45:40.746905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 06 14:45:40.746905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 06 14:45:40.746905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:40.746905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 06 14:45:40.746905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:40.746905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 06 14:45:40.746905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:40.746905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:40.746905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:40.746905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:40.746905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:40.746905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:40.747909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:40.747909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:40.747909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:40.747909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:40.747909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:40.747909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:40.747909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 06 14:45:40.747909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(service_id) May 06 14:45:40.747909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 06 14:45:40.747909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 06 14:45:40.747909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:40.747909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:40.747909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:40.747909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:40.747909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:40.747909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:40.748791 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:40.748791 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:40.748791 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:40.748791 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:40.748791 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:40.748791 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:40.748791 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:40.748791 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:40.748791 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:40.748791 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:40.748791 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 06 14:45:40.748791 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:40.748791 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 06 14:45:40.748791 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:40.752187 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 454/915] 10.222.0.23 () {56 vars in 1114 bytes} [Fri May 6 14:45:40 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:40.754825 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad6cf84-931b-4d82-9cfa-b68926aad215 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:40.856739 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad6cf84-931b-4d82-9cfa-b68926aad215 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:40.896026 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc5fe5-b6d1-4a4b-9b52-abc4c066b43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:40.896734 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc5fe5-b6d1-4a4b-9b52-abc4c066b43b None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:40.897024 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc5fe5-b6d1-4a4b-9b52-abc4c066b43b None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:40.897302 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc5fe5-b6d1-4a4b-9b52-abc4c066b43b None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:40.898011 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cc5fe5-b6d1-4a4b-9b52-abc4c066b43b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:40.898319 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cc5fe5-b6d1-4a4b-9b52-abc4c066b43b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:40.899689 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cc5fe5-b6d1-4a4b-9b52-abc4c066b43b None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:40.908733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 462/916] 10.222.0.23 () {56 vars in 1114 bytes} [Fri May 6 14:45:40 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:40.911143 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b640a27-6e47-4389-a2ea-95d5bbd54316 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:41.014634 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b640a27-6e47-4389-a2ea-95d5bbd54316 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:41.050413 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c546a8-26bd-438f-9248-87d5294b7bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:41.051753 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c546a8-26bd-438f-9248-87d5294b7bba None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:41.052051 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c546a8-26bd-438f-9248-87d5294b7bba None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:41.052348 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c546a8-26bd-438f-9248-87d5294b7bba None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:41.053165 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c546a8-26bd-438f-9248-87d5294b7bba None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:41.053475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c546a8-26bd-438f-9248-87d5294b7bba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:41.054856 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c546a8-26bd-438f-9248-87d5294b7bba None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:41.064892 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 455/917] 10.222.0.23 () {56 vars in 1114 bytes} [Fri May 6 14:45:40 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:41.067992 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc5fe5-b6d1-4a4b-9b52-abc4c066b43b None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:41.188897 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc5fe5-b6d1-4a4b-9b52-abc4c066b43b None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:41.224731 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec32b3d-8711-45c7-957f-6ed7c67ff8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:41.225404 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec32b3d-8711-45c7-957f-6ed7c67ff8c6 None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:41.225690 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec32b3d-8711-45c7-957f-6ed7c67ff8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:41.225969 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec32b3d-8711-45c7-957f-6ed7c67ff8c6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:41.226720 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec32b3d-8711-45c7-957f-6ed7c67ff8c6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:41.228082 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec32b3d-8711-45c7-957f-6ed7c67ff8c6 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:41.252809 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 463/918] 10.222.0.23 () {60 vars in 1144 bytes} [Fri May 6 14:45:41 2022] POST /identity/v3/endpoints => generated 346 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 14:45:41.262221 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce95f61-85a9-4c02-bbb8-613909d8f972 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:41.262577 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce95f61-85a9-4c02-bbb8-613909d8f972 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:41.262867 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce95f61-85a9-4c02-bbb8-613909d8f972 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:41.264919 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 456/919] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:45:41.271042 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e8f6f-8c00-48e7-b97b-3565bfeba739 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:41.271437 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e8f6f-8c00-48e7-b97b-3565bfeba739 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:41.271726 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e8f6f-8c00-48e7-b97b-3565bfeba739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:41.291584 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-9e4e8f6f-8c00-48e7-b97b-3565bfeba739 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:41.292867 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-9e4e8f6f-8c00-48e7-b97b-3565bfeba739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:41.384572 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 464/920] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:45:41.407285 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce95f61-85a9-4c02-bbb8-613909d8f972 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:41.526499 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce95f61-85a9-4c02-bbb8-613909d8f972 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:41.561502 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2880d1f1-4034-44b6-8010-b6d4cc9cfe38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:41.562354 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2880d1f1-4034-44b6-8010-b6d4cc9cfe38 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:41.562636 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2880d1f1-4034-44b6-8010-b6d4cc9cfe38 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:41.562908 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2880d1f1-4034-44b6-8010-b6d4cc9cfe38 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:41.563557 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2880d1f1-4034-44b6-8010-b6d4cc9cfe38 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:41.564954 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2880d1f1-4034-44b6-8010-b6d4cc9cfe38 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:41.577320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-2880d1f1-4034-44b6-8010-b6d4cc9cfe38 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 06 14:45:41.577320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:41.577320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 06 14:45:41.577320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 06 14:45:41.577320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 06 14:45:41.577320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 06 14:45:41.577320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 06 14:45:41.577320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:41.577320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 06 14:45:41.577320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:41.577320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 06 14:45:41.577320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:41.577320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:41.577320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:41.577320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:41.577320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:41.577320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:41.579692 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:41.579692 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:41.579692 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:41.579692 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:41.579692 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:41.579692 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:41.579692 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 06 14:45:41.579692 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(service_id) May 06 14:45:41.579692 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 06 14:45:41.579692 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 06 14:45:41.579692 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:41.579692 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:41.579692 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:41.579692 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:41.579692 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:41.579692 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:41.581766 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:41.581766 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:41.581766 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:41.581766 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:41.581766 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:41.581766 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:41.581766 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:41.581766 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:41.581766 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:41.581766 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:41.581766 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 06 14:45:41.581766 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:41.581766 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 06 14:45:41.581766 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:41.581766 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 457/921] 10.222.0.23 () {56 vars in 1126 bytes} [Fri May 6 14:45:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:41.587581 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e8f6f-8c00-48e7-b97b-3565bfeba739 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:41.663296 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32c010-6c3e-421f-9ce7-573df2ad367c None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:41.663631 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32c010-6c3e-421f-9ce7-573df2ad367c None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:41.663851 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32c010-6c3e-421f-9ce7-573df2ad367c None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:41.665566 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 458/922] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:45:41.673725 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08360b28-e1c0-48c2-b79b-85945ddb8d0e None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:41.674029 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08360b28-e1c0-48c2-b79b-85945ddb8d0e None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:41.674275 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08360b28-e1c0-48c2-b79b-85945ddb8d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:41.693391 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-08360b28-e1c0-48c2-b79b-85945ddb8d0e None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:41.694576 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-08360b28-e1c0-48c2-b79b-85945ddb8d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:41.699349 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e8f6f-8c00-48e7-b97b-3565bfeba739 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:41.734833 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bb73b4-41ae-4715-8d7d-25d98fd239bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:41.735891 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb73b4-41ae-4715-8d7d-25d98fd239bb None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:41.736375 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb73b4-41ae-4715-8d7d-25d98fd239bb None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:41.736894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb73b4-41ae-4715-8d7d-25d98fd239bb None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:41.737793 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bb73b4-41ae-4715-8d7d-25d98fd239bb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:41.738341 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bb73b4-41ae-4715-8d7d-25d98fd239bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:41.740652 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bb73b4-41ae-4715-8d7d-25d98fd239bb None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:41.746083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 465/923] 10.222.0.23 () {56 vars in 1122 bytes} [Fri May 6 14:45:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:41.751789 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bb73b4-41ae-4715-8d7d-25d98fd239bb None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:41.772723 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 459/924] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 06 14:45:41.789571 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08360b28-e1c0-48c2-b79b-85945ddb8d0e None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:41.854117 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bb73b4-41ae-4715-8d7d-25d98fd239bb None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:41.891128 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12751a7f-1d2f-49fe-8bb2-f31de6a95d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:41.891360 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08360b28-e1c0-48c2-b79b-85945ddb8d0e None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:41.892313 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12751a7f-1d2f-49fe-8bb2-f31de6a95d23 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:41.892833 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12751a7f-1d2f-49fe-8bb2-f31de6a95d23 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:41.893407 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12751a7f-1d2f-49fe-8bb2-f31de6a95d23 None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:41.894408 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12751a7f-1d2f-49fe-8bb2-f31de6a95d23 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:41.895029 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12751a7f-1d2f-49fe-8bb2-f31de6a95d23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:41.896677 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12751a7f-1d2f-49fe-8bb2-f31de6a95d23 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:41.907373 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 466/925] 10.222.0.23 () {56 vars in 1122 bytes} [Fri May 6 14:45:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:41.908655 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12751a7f-1d2f-49fe-8bb2-f31de6a95d23 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:41.928523 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3d6f6-cbed-4260-952e-a62a7895f7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:41.929372 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3d6f6-cbed-4260-952e-a62a7895f7d4 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:41.929651 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3d6f6-cbed-4260-952e-a62a7895f7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:41.929924 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3d6f6-cbed-4260-952e-a62a7895f7d4 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:41.930567 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba3d6f6-cbed-4260-952e-a62a7895f7d4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:41.931918 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba3d6f6-cbed-4260-952e-a62a7895f7d4 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:41.944639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-bba3d6f6-cbed-4260-952e-a62a7895f7d4 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 06 14:45:41.944639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:41.944639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 06 14:45:41.944639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 06 14:45:41.944639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 06 14:45:41.944639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 06 14:45:41.944639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 06 14:45:41.944639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:41.944639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 06 14:45:41.944639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:41.944639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 06 14:45:41.944639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:41.944639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:41.944639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:41.944639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:41.944639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:41.945605 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:41.945605 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:41.945605 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:41.945605 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:41.945605 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:41.945605 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:41.945605 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:41.945605 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 06 14:45:41.945605 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(service_id) May 06 14:45:41.945605 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 06 14:45:41.945605 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 06 14:45:41.945605 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:41.945605 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:41.945605 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:41.945605 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:41.945605 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:41.946458 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:41.946458 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:41.946458 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:41.946458 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:41.946458 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:41.946458 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:41.946458 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:41.946458 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:41.946458 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:41.946458 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:41.946458 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:41.946458 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 06 14:45:41.946458 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:41.946458 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 06 14:45:41.946458 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:41.950857 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 460/926] 10.222.0.23 () {56 vars in 1132 bytes} [Fri May 6 14:45:41 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:45:41.952150 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3d6f6-cbed-4260-952e-a62a7895f7d4 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:42.014715 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12751a7f-1d2f-49fe-8bb2-f31de6a95d23 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:42.050891 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb7b35f-661d-4c4b-a69d-8295d593cc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:42.052523 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7b35f-661d-4c4b-a69d-8295d593cc58 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:42.053042 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7b35f-661d-4c4b-a69d-8295d593cc58 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:42.053538 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7b35f-661d-4c4b-a69d-8295d593cc58 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:42.054518 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb7b35f-661d-4c4b-a69d-8295d593cc58 None admin] RBAC: Adding query filter params (interface=public, service_id=4af42964328248c7bea104867d364c8d, region_id=RegionOne) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:42.055044 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb7b35f-661d-4c4b-a69d-8295d593cc58 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:42.056676 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb7b35f-661d-4c4b-a69d-8295d593cc58 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:42.056949 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3d6f6-cbed-4260-952e-a62a7895f7d4 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:42.068107 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 467/927] 10.222.0.23 () {56 vars in 1251 bytes} [Fri May 6 14:45:41 2022] GET /identity/v3/endpoints?service_id=4af42964328248c7bea104867d364c8d&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:42.069398 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb7b35f-661d-4c4b-a69d-8295d593cc58 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:42.093661 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf4a15-6d38-4b9d-b6e3-664ae254fb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:42.094476 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf4a15-6d38-4b9d-b6e3-664ae254fb45 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:42.094682 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf4a15-6d38-4b9d-b6e3-664ae254fb45 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:42.094975 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf4a15-6d38-4b9d-b6e3-664ae254fb45 None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:42.095718 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf4a15-6d38-4b9d-b6e3-664ae254fb45 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:42.096338 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf4a15-6d38-4b9d-b6e3-664ae254fb45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:42.097555 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf4a15-6d38-4b9d-b6e3-664ae254fb45 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:42.107986 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 461/928] 10.222.0.23 () {56 vars in 1126 bytes} [Fri May 6 14:45:41 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:42.109249 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf4a15-6d38-4b9d-b6e3-664ae254fb45 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:42.173745 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb7b35f-661d-4c4b-a69d-8295d593cc58 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:42.209474 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb0b26-6db9-4362-a136-a394c9fbce1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:42.210316 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb0b26-6db9-4362-a136-a394c9fbce1b None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:42.210700 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb0b26-6db9-4362-a136-a394c9fbce1b None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:42.211080 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb0b26-6db9-4362-a136-a394c9fbce1b None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:42.211865 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eb0b26-6db9-4362-a136-a394c9fbce1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:42.213352 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eb0b26-6db9-4362-a136-a394c9fbce1b None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:42.214321 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf4a15-6d38-4b9d-b6e3-664ae254fb45 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:42.224906 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 468/929] 10.222.0.23 () {56 vars in 1087 bytes} [Fri May 6 14:45:42 2022] GET /identity/v3/services => generated 1497 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:45:42.252428 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf6aaf5-774b-4232-895d-72f6027a30f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:42.253128 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6aaf5-774b-4232-895d-72f6027a30f2 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:42.253406 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6aaf5-774b-4232-895d-72f6027a30f2 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:42.253677 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6aaf5-774b-4232-895d-72f6027a30f2 None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:42.255051 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf6aaf5-774b-4232-895d-72f6027a30f2 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:42.255368 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf6aaf5-774b-4232-895d-72f6027a30f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:42.256805 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf6aaf5-774b-4232-895d-72f6027a30f2 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:42.267192 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 462/930] 10.222.0.23 () {56 vars in 1126 bytes} [Fri May 6 14:45:42 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:42.268746 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb0b26-6db9-4362-a136-a394c9fbce1b None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:42.381058 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb0b26-6db9-4362-a136-a394c9fbce1b None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:42.418036 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c45f279-0e52-4afa-b8cb-35d561e4c839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:42.418707 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c45f279-0e52-4afa-b8cb-35d561e4c839 None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:42.418986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c45f279-0e52-4afa-b8cb-35d561e4c839 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:42.419260 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c45f279-0e52-4afa-b8cb-35d561e4c839 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:42.420015 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c45f279-0e52-4afa-b8cb-35d561e4c839 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:42.421427 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c45f279-0e52-4afa-b8cb-35d561e4c839 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:42.452717 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 469/931] 10.222.0.23 () {60 vars in 1144 bytes} [Fri May 6 14:45:42 2022] POST /identity/v3/endpoints => generated 347 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:45:42.693103 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f5b41-e281-4425-b5f7-3dcce1c4cc27 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:42.693764 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f5b41-e281-4425-b5f7-3dcce1c4cc27 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:42.693764 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f5b41-e281-4425-b5f7-3dcce1c4cc27 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:42.695223 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 463/932] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:45:42.700659 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9dded2-ac2d-4167-a2af-82f78a2bffbd None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:42.700958 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9dded2-ac2d-4167-a2af-82f78a2bffbd None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:42.701239 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9dded2-ac2d-4167-a2af-82f78a2bffbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:42.718986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-da9dded2-ac2d-4167-a2af-82f78a2bffbd None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:42.720155 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-da9dded2-ac2d-4167-a2af-82f78a2bffbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:42.794152 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 470/933] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:45:42.817585 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f5b41-e281-4425-b5f7-3dcce1c4cc27 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:42.919127 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f5b41-e281-4425-b5f7-3dcce1c4cc27 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:42.954115 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78524efe-92fd-4a2a-9808-ee726b734b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:42.954907 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78524efe-92fd-4a2a-9808-ee726b734b31 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:42.955185 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78524efe-92fd-4a2a-9808-ee726b734b31 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:42.955505 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78524efe-92fd-4a2a-9808-ee726b734b31 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:42.966274 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78524efe-92fd-4a2a-9808-ee726b734b31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:45:42.974577 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-78524efe-92fd-4a2a-9808-ee726b734b31 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 06 14:45:42.974577 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:42.974577 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:42.974577 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:42.974577 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:42.974577 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:42.974577 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:42.974577 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:42.974577 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:42.974577 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:42.974577 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:42.974577 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:42.974577 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:45:42.974577 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:45:42.974577 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:45:42.974577 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:45:42.976755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:45:42.976755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:45:42.976755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:45:42.976755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:45:42.976755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:42.976755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:42.976755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:42.976755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:42.976755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:42.976755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:42.976755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:42.976755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:42.976755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:42.976755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:42.976755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:42.976755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:42.978775 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:42.978775 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:42.978775 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:42.978775 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:42.978775 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:45:42.978775 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:45:42.978775 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:45:42.978775 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:45:42.978775 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:45:42.978775 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:45:42.978775 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 06 14:45:42.978775 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:42.978775 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 464/934] 10.222.0.23 () {56 vars in 1120 bytes} [Fri May 6 14:45:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:42.984637 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9dded2-ac2d-4167-a2af-82f78a2bffbd None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:43.083912 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9dded2-ac2d-4167-a2af-82f78a2bffbd None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:43.123122 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6b7c65-8b7c-4114-802b-68414bf0b850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:43.123799 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b7c65-8b7c-4114-802b-68414bf0b850 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:43.124077 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b7c65-8b7c-4114-802b-68414bf0b850 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:43.124347 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b7c65-8b7c-4114-802b-68414bf0b850 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:43.125133 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6b7c65-8b7c-4114-802b-68414bf0b850 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:43.125432 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6b7c65-8b7c-4114-802b-68414bf0b850 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:43.127058 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6b7c65-8b7c-4114-802b-68414bf0b850 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:43.137018 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 471/935] 10.222.0.23 () {56 vars in 1115 bytes} [Fri May 6 14:45:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:43.142808 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78524efe-92fd-4a2a-9808-ee726b734b31 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:43.256276 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78524efe-92fd-4a2a-9808-ee726b734b31 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:43.291662 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b897ac-ff56-4796-b408-ada38d6b8948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:43.292447 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b897ac-ff56-4796-b408-ada38d6b8948 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:43.292752 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b897ac-ff56-4796-b408-ada38d6b8948 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:43.293026 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b897ac-ff56-4796-b408-ada38d6b8948 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:43.304915 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b897ac-ff56-4796-b408-ada38d6b8948 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:43.306344 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b897ac-ff56-4796-b408-ada38d6b8948 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:43.323935 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-f8b897ac-ff56-4796-b408-ada38d6b8948 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:43.323935 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:43.323935 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:43.323935 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:43.323935 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:43.323935 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:43.323935 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:43.323935 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:43.323935 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:43.323935 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:43.323935 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:43.323935 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:43.323935 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:43.323935 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:43.323935 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:43.324876 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:43.324876 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:43.324876 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:43.324876 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:43.324876 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:43.324876 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:43.324876 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:43.324876 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:43.324876 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:43.324876 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:43.324876 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:43.324876 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:43.324876 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:43.324876 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:43.324876 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:43.324876 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:43.324876 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:43.326257 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:43.326257 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:43.326257 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:43.326257 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:43.326257 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 465/936] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:43.331301 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6b7c65-8b7c-4114-802b-68414bf0b850 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:43.440773 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6b7c65-8b7c-4114-802b-68414bf0b850 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:43.479675 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a580e9d-a4c0-4077-baf4-6b27b7e9d2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:43.480353 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a580e9d-a4c0-4077-baf4-6b27b7e9d2d6 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:43.480658 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a580e9d-a4c0-4077-baf4-6b27b7e9d2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:43.480940 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a580e9d-a4c0-4077-baf4-6b27b7e9d2d6 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:43.481668 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a580e9d-a4c0-4077-baf4-6b27b7e9d2d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:43.481973 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a580e9d-a4c0-4077-baf4-6b27b7e9d2d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:43.483359 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a580e9d-a4c0-4077-baf4-6b27b7e9d2d6 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:43.497016 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 472/937] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:43.502248 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b897ac-ff56-4796-b408-ada38d6b8948 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:43.609566 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b897ac-ff56-4796-b408-ada38d6b8948 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:43.646768 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e01b9f-5031-4fe1-a2f3-5b686995183d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:43.647109 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3f08a-9c7e-471e-bd4e-274478bea14f None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:43.647286 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3f08a-9c7e-471e-bd4e-274478bea14f None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:43.647431 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3f08a-9c7e-471e-bd4e-274478bea14f None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:43.647571 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e01b9f-5031-4fe1-a2f3-5b686995183d None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:43.647877 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e01b9f-5031-4fe1-a2f3-5b686995183d None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:43.648148 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e01b9f-5031-4fe1-a2f3-5b686995183d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:43.649073 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 473/938] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:45:43.654020 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81054eb0-b2ab-4655-817c-52b4e7954b6a None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:43.654310 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81054eb0-b2ab-4655-817c-52b4e7954b6a None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:43.654593 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81054eb0-b2ab-4655-817c-52b4e7954b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:43.655954 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e01b9f-5031-4fe1-a2f3-5b686995183d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:43.657414 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e01b9f-5031-4fe1-a2f3-5b686995183d None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:43.672116 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-81054eb0-b2ab-4655-817c-52b4e7954b6a None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:43.673325 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-48e01b9f-5031-4fe1-a2f3-5b686995183d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 06 14:45:43.673325 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:43.673325 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:43.673325 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:43.673325 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:43.673325 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:43.673325 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:43.673325 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:43.673325 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:43.673325 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:43.673325 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:43.673325 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:43.673325 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 06 14:45:43.673325 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_user(user_id) May 06 14:45:43.673325 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 06 14:45:43.673325 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 06 14:45:43.675337 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:43.675337 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:43.675337 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 06 14:45:43.675337 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:43.675337 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 06 14:45:43.675337 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:43.675337 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:43.675337 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:43.675337 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:43.675337 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:43.675337 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:43.675337 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:43.675337 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:43.675337 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:43.675337 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:43.675337 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:43.675337 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:43.675337 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:43.677198 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:43.677198 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:43.677198 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 06 14:45:43.677198 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 06 14:45:43.677198 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 06 14:45:43.677198 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 06 14:45:43.677198 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 06 14:45:43.677198 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 06 14:45:43.677198 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 06 14:45:43.677198 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:43.677962 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-81054eb0-b2ab-4655-817c-52b4e7954b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:43.678058 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 466/939] 10.222.0.23 () {56 vars in 1117 bytes} [Fri May 6 14:45:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:43.686066 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e01b9f-5031-4fe1-a2f3-5b686995183d None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:43.752766 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 474/940] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 06 14:45:43.772987 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81054eb0-b2ab-4655-817c-52b4e7954b6a None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:43.810150 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e01b9f-5031-4fe1-a2f3-5b686995183d None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:43.847017 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f7961-2bd8-4d96-9b7b-237beef6cf6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:43.847693 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f7961-2bd8-4d96-9b7b-237beef6cf6e None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:43.848007 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f7961-2bd8-4d96-9b7b-237beef6cf6e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:43.848282 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f7961-2bd8-4d96-9b7b-237beef6cf6e None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:43.849155 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581f7961-2bd8-4d96-9b7b-237beef6cf6e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:43.849466 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581f7961-2bd8-4d96-9b7b-237beef6cf6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:43.850884 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581f7961-2bd8-4d96-9b7b-237beef6cf6e None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:43.871429 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 467/941] 10.222.0.23 () {56 vars in 1149 bytes} [Fri May 6 14:45:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:43.876502 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f7961-2bd8-4d96-9b7b-237beef6cf6e None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:43.902512 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81054eb0-b2ab-4655-817c-52b4e7954b6a None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:43.948981 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5f812-a2ed-4918-af18-a8fcc6ba684d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:43.949675 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5f812-a2ed-4918-af18-a8fcc6ba684d None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:43.949967 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5f812-a2ed-4918-af18-a8fcc6ba684d None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:43.950249 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5f812-a2ed-4918-af18-a8fcc6ba684d None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:43.951003 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc5f812-a2ed-4918-af18-a8fcc6ba684d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:43.952422 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc5f812-a2ed-4918-af18-a8fcc6ba684d None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:43.954348 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: WARNING keystone.server.flask.common [None req-ddc5f812-a2ed-4918-af18-a8fcc6ba684d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 06 14:45:43.977305 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.sql.core [None req-ddc5f812-a2ed-4918-af18-a8fcc6ba684d None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 06 14:45:43.977305 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 06 14:45:43.977305 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [parameters: {'id': 'e36002eb454d4f5d94375b86324c0352', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 06 14:45:43.977305 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=79888) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 06 14:45:43.984534 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f7961-2bd8-4d96-9b7b-237beef6cf6e None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:43.984953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-ddc5f812-a2ed-4918-af18-a8fcc6ba684d None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e36002eb454d4f5d94375b86324c0352.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e36002eb454d4f5d94375b86324c0352. May 06 14:45:43.984953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:43.984953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 06 14:45:43.984953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self.dialect.do_execute( May 06 14:45:43.984953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 06 14:45:43.984953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 06 14:45:43.984953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 06 14:45:43.984953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application result = self._query(query) May 06 14:45:43.984953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 06 14:45:43.984953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application conn.query(q) May 06 14:45:43.984953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 06 14:45:43.984953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 06 14:45:43.984953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 06 14:45:43.986796 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application result.read() May 06 14:45:43.986796 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 06 14:45:43.986796 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 06 14:45:43.986796 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 06 14:45:43.986796 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application packet.raise_for_error() May 06 14:45:43.986796 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 06 14:45:43.986796 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 06 14:45:43.986796 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 06 14:45:43.986796 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 06 14:45:43.986796 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 06 14:45:43.986796 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:43.986796 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 06 14:45:43.986796 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:43.986796 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:43.986796 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 06 14:45:43.986796 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 06 14:45:43.986796 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 06 14:45:43.986796 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return project_ref.to_dict() May 06 14:45:43.989803 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 06 14:45:43.989803 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application next(self.gen) May 06 14:45:43.989803 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 06 14:45:43.989803 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application yield resource May 06 14:45:43.989803 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 06 14:45:43.989803 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application next(self.gen) May 06 14:45:43.989803 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 06 14:45:43.989803 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self.session.rollback() May 06 14:45:43.989803 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 06 14:45:43.989803 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self.force_reraise() May 06 14:45:43.989803 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 06 14:45:43.989803 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise self.value May 06 14:45:43.989803 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 06 14:45:43.989803 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 06 14:45:43.989803 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 06 14:45:43.989803 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application session.commit() May 06 14:45:43.989803 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 06 14:45:43.993083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 06 14:45:43.993083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 06 14:45:43.993083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._prepare_impl() May 06 14:45:43.993083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 06 14:45:43.993083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self.session.flush() May 06 14:45:43.993083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 06 14:45:43.993083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._flush(objects) May 06 14:45:43.993083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 06 14:45:43.993083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 06 14:45:43.993083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 06 14:45:43.993083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application compat.raise_( May 06 14:45:43.993083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 06 14:45:43.993083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception May 06 14:45:43.993083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 06 14:45:43.993083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application flush_context.execute() May 06 14:45:43.993083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 06 14:45:43.993083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rec.execute(self) May 06 14:45:43.994007 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 06 14:45:43.994007 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 06 14:45:43.994007 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 06 14:45:43.994007 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application _emit_insert_statements( May 06 14:45:43.994007 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 06 14:45:43.994007 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application c = connection._execute_20( May 06 14:45:43.994007 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 06 14:45:43.994007 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 06 14:45:43.994007 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 06 14:45:43.994007 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 06 14:45:43.994007 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 06 14:45:43.994007 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ret = self._execute_context( May 06 14:45:43.994007 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 06 14:45:43.994007 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 06 14:45:43.994007 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 06 14:45:43.994839 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 06 14:45:43.994839 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 06 14:45:43.994839 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception May 06 14:45:43.994839 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 06 14:45:43.994839 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self.dialect.do_execute( May 06 14:45:43.994839 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 06 14:45:43.994839 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 06 14:45:43.994839 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 06 14:45:43.994839 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application result = self._query(query) May 06 14:45:43.994839 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 06 14:45:43.994839 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application conn.query(q) May 06 14:45:43.994839 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 06 14:45:43.994839 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 06 14:45:43.994839 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 06 14:45:43.994839 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application result.read() May 06 14:45:43.994839 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 06 14:45:43.994839 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 06 14:45:43.995714 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 06 14:45:43.995714 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application packet.raise_for_error() May 06 14:45:43.995714 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 06 14:45:43.995714 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 06 14:45:43.995714 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 06 14:45:43.995714 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 06 14:45:43.995714 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 06 14:45:43.995714 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 06 14:45:43.995714 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [parameters: {'id': 'e36002eb454d4f5d94375b86324c0352', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 06 14:45:43.995714 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 06 14:45:43.995714 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:43.995714 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 06 14:45:43.995714 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:43.995714 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:43.995714 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 06 14:45:43.996534 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 06 14:45:43.996534 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 06 14:45:43.996534 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 06 14:45:43.996534 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 06 14:45:43.996534 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:43.996534 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 06 14:45:43.996534 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:43.996534 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:43.996534 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:43.996534 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:43.996534 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:43.996534 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:43.996534 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:43.996534 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:43.996534 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:43.996534 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:43.996534 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:43.998373 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:43.998373 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 06 14:45:43.998373 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 06 14:45:43.998373 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:43.998373 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:43.998373 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 06 14:45:43.998373 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.Conflict( May 06 14:45:43.998373 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e36002eb454d4f5d94375b86324c0352. May 06 14:45:43.998373 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:43.998373 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 475/942] 10.222.0.23 () {60 vars in 1140 bytes} [Fri May 6 14:45:43 2022] POST /identity/v3/projects => generated 262 bytes in 219 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 06 14:45:44.000638 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5f812-a2ed-4918-af18-a8fcc6ba684d None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:44.028954 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ce65b-868b-4e4e-b09a-d2c8561d7a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:44.030173 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ce65b-868b-4e4e-b09a-d2c8561d7a52 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:44.030891 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ce65b-868b-4e4e-b09a-d2c8561d7a52 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:44.031344 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ce65b-868b-4e4e-b09a-d2c8561d7a52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:44.043289 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ce65b-868b-4e4e-b09a-d2c8561d7a52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:44.045107 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ce65b-868b-4e4e-b09a-d2c8561d7a52 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:44.064819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-c06ce65b-868b-4e4e-b09a-d2c8561d7a52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:44.064819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:44.064819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:44.064819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:44.064819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:44.064819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:44.064819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:44.064819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:44.064819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:44.064819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:44.064819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:44.064819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:44.064819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:44.064819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:44.064819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:44.065789 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:44.065789 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:44.065789 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:44.065789 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:44.065789 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:44.065789 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:44.065789 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:44.065789 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:44.065789 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:44.065789 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:44.065789 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:44.065789 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:44.065789 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:44.065789 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:44.065789 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:44.065789 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:44.065789 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:44.066951 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:44.066951 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:44.066951 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:44.066951 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:44.069075 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 468/943] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:44.074554 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ce65b-868b-4e4e-b09a-d2c8561d7a52 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:44.115300 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5f812-a2ed-4918-af18-a8fcc6ba684d None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:44.150575 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb622b-62ef-4a37-99fe-f628bd573391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:44.151625 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb622b-62ef-4a37-99fe-f628bd573391 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:44.152044 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb622b-62ef-4a37-99fe-f628bd573391 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:44.152467 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb622b-62ef-4a37-99fe-f628bd573391 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:44.159958 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffb622b-62ef-4a37-99fe-f628bd573391 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:44.162364 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffb622b-62ef-4a37-99fe-f628bd573391 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:44.181611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-1ffb622b-62ef-4a37-99fe-f628bd573391 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:44.181611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:44.181611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:44.181611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:44.181611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:44.181611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:44.181611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:44.181611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:44.181611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:44.181611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:44.181611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:44.181611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:44.181611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 06 14:45:44.181611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(project_id) May 06 14:45:44.181611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 06 14:45:44.182553 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 06 14:45:44.182553 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:44.182553 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:44.182553 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:44.182553 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:44.182553 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:44.182553 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:44.182553 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:44.182553 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:44.182553 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:44.182553 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:44.182553 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:44.182553 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:44.182553 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:44.182553 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:44.182553 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:44.182553 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:44.183449 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 06 14:45:44.183449 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 06 14:45:44.183449 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 06 14:45:44.183449 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 06 14:45:44.183449 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 06 14:45:44.183449 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 06 14:45:44.183449 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:44.183449 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:44.190438 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 476/944] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:45:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:44.191590 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb622b-62ef-4a37-99fe-f628bd573391 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:44.197226 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ce65b-868b-4e4e-b09a-d2c8561d7a52 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:44.243208 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faad666b-f7f8-4b7b-b226-4a8ab34b4df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:44.243869 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad666b-f7f8-4b7b-b226-4a8ab34b4df6 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:44.244150 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad666b-f7f8-4b7b-b226-4a8ab34b4df6 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:44.244731 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad666b-f7f8-4b7b-b226-4a8ab34b4df6 None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:44.245466 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faad666b-f7f8-4b7b-b226-4a8ab34b4df6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:44.245786 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faad666b-f7f8-4b7b-b226-4a8ab34b4df6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:44.247125 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faad666b-f7f8-4b7b-b226-4a8ab34b4df6 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:44.264697 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 469/945] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:44.268219 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faad666b-f7f8-4b7b-b226-4a8ab34b4df6 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:44.302847 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb622b-62ef-4a37-99fe-f628bd573391 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:44.338316 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e48bc-0652-4027-84cb-c19623361892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:44.338969 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e48bc-0652-4027-84cb-c19623361892 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:44.339265 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e48bc-0652-4027-84cb-c19623361892 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:44.339509 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e48bc-0652-4027-84cb-c19623361892 None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:44.340258 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e48bc-0652-4027-84cb-c19623361892 None admin] RBAC: Adding query filter params (name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:44.340565 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e48bc-0652-4027-84cb-c19623361892 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:44.342032 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e48bc-0652-4027-84cb-c19623361892 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:44.363347 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 477/946] 10.222.0.23 () {56 vars in 1112 bytes} [Fri May 6 14:45:44 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:44.393869 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faad666b-f7f8-4b7b-b226-4a8ab34b4df6 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:44.437515 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db383f9-5fa2-46c8-b346-2899803ca56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:44.438710 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db383f9-5fa2-46c8-b346-2899803ca56e None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:44.439217 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db383f9-5fa2-46c8-b346-2899803ca56e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:44.439695 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db383f9-5fa2-46c8-b346-2899803ca56e None admin] PATH_INFO: `/v3/projects/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:44.448501 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db383f9-5fa2-46c8-b346-2899803ca56e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:44.450305 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db383f9-5fa2-46c8-b346-2899803ca56e None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:44.466109 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-9db383f9-5fa2-46c8-b346-2899803ca56e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:44.466109 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:44.466109 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:44.466109 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:44.466109 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:44.466109 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:44.466109 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:44.466109 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:44.466109 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:44.466109 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:44.466109 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:44.466109 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:44.466109 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 06 14:45:44.466109 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(project_id) May 06 14:45:44.466109 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 06 14:45:44.467267 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 06 14:45:44.467267 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:44.467267 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:44.467267 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:44.467267 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:44.467267 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:44.467267 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:44.467267 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:44.467267 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:44.467267 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:44.467267 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:44.467267 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:44.467267 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:44.467267 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:44.467267 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:44.467267 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:44.467267 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:44.468392 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 06 14:45:44.468392 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 06 14:45:44.468392 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 06 14:45:44.468392 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 06 14:45:44.468392 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 06 14:45:44.468392 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 06 14:45:44.468392 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:44.468392 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:44.470625 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 470/947] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:45:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:44.476878 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e48bc-0652-4027-84cb-c19623361892 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:44.586100 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e48bc-0652-4027-84cb-c19623361892 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:44.601355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454e74c-fcd3-41c0-9cab-89d4c30b6a42 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:44.601987 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454e74c-fcd3-41c0-9cab-89d4c30b6a42 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:44.602330 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454e74c-fcd3-41c0-9cab-89d4c30b6a42 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:44.604012 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 471/948] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:45:44.609921 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d9758-1e7d-4092-b6e3-4b2974863d49 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:44.610259 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d9758-1e7d-4092-b6e3-4b2974863d49 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:44.610622 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d9758-1e7d-4092-b6e3-4b2974863d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:44.622267 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06387017-9b3c-4a85-a649-7fd3c0c0e652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:44.622937 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06387017-9b3c-4a85-a649-7fd3c0c0e652 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:44.623213 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06387017-9b3c-4a85-a649-7fd3c0c0e652 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:44.623483 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06387017-9b3c-4a85-a649-7fd3c0c0e652 None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:44.624258 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06387017-9b3c-4a85-a649-7fd3c0c0e652 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:44.624665 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06387017-9b3c-4a85-a649-7fd3c0c0e652 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:44.626341 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06387017-9b3c-4a85-a649-7fd3c0c0e652 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:44.629852 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-9a5d9758-1e7d-4092-b6e3-4b2974863d49 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:44.631061 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-9a5d9758-1e7d-4092-b6e3-4b2974863d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:44.639563 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 478/949] 10.222.0.23 () {56 vars in 1148 bytes} [Fri May 6 14:45:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:44.646329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06387017-9b3c-4a85-a649-7fd3c0c0e652 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:44.717391 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 472/950] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 06 14:45:44.743346 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d9758-1e7d-4092-b6e3-4b2974863d49 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:44.764742 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06387017-9b3c-4a85-a649-7fd3c0c0e652 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:44.802543 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d59478-842e-44cb-8c3c-8a84f9decacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:44.803442 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d59478-842e-44cb-8c3c-8a84f9decacd None admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:44.803797 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d59478-842e-44cb-8c3c-8a84f9decacd None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:44.804130 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d59478-842e-44cb-8c3c-8a84f9decacd None admin] PATH_INFO: `/v3/projects/d5398feee6eb490cb49bb261448a1a9b/users/a7dc1bc53c59423e934f7831ea022c73/roles/5213da7472024e87a41cecfaa37abce3` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:44.806539 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d59478-842e-44cb-8c3c-8a84f9decacd None admin] RBAC: Authorizing `identity:create_grant(project_id=d5398feee6eb490cb49bb261448a1a9b, user_id=a7dc1bc53c59423e934f7831ea022c73, role_id=5213da7472024e87a41cecfaa37abce3)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:44.808016 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d59478-842e-44cb-8c3c-8a84f9decacd None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:44.826498 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 479/951] 10.222.0.23 () {58 vars in 1439 bytes} [Fri May 6 14:45:44 2022] PUT /identity/v3/projects/d5398feee6eb490cb49bb261448a1a9b/users/a7dc1bc53c59423e934f7831ea022c73/roles/5213da7472024e87a41cecfaa37abce3 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 14:45:44.855858 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d9758-1e7d-4092-b6e3-4b2974863d49 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:44.897635 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09018d3d-e732-42a2-a73c-365b183d5ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:44.898524 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09018d3d-e732-42a2-a73c-365b183d5ebe None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:44.898812 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09018d3d-e732-42a2-a73c-365b183d5ebe None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:44.899089 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09018d3d-e732-42a2-a73c-365b183d5ebe None admin] PATH_INFO: `/v3/services/object-store` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:44.899747 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09018d3d-e732-42a2-a73c-365b183d5ebe None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:44.901239 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09018d3d-e732-42a2-a73c-365b183d5ebe None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:44.915070 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-09018d3d-e732-42a2-a73c-365b183d5ebe None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 06 14:45:44.915070 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:44.915070 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 06 14:45:44.915070 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 06 14:45:44.915070 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 06 14:45:44.915070 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 06 14:45:44.915070 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 06 14:45:44.915070 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:44.915070 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 06 14:45:44.915070 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:44.915070 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 06 14:45:44.915070 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:44.915070 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:44.915070 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:44.915070 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:44.915070 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:44.915070 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:44.916138 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:44.916138 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:44.916138 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:44.916138 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:44.916138 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:44.916138 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:44.916138 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 06 14:45:44.916138 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(service_id) May 06 14:45:44.916138 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 06 14:45:44.916138 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 06 14:45:44.916138 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:44.916138 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:44.916138 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:44.916138 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:44.916138 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:44.916138 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:44.917024 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:44.917024 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:44.917024 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:44.917024 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:44.917024 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:44.917024 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:44.917024 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:44.917024 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:44.917024 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:44.917024 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:44.917024 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 06 14:45:44.917024 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:44.917024 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 06 14:45:44.917024 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:44.920820 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 473/952] 10.222.0.23 () {56 vars in 1126 bytes} [Fri May 6 14:45:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:44.926127 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d59478-842e-44cb-8c3c-8a84f9decacd None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:44.927827 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760e484-f0d2-4520-beae-37ae0c85688f None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:44.928123 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760e484-f0d2-4520-beae-37ae0c85688f None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:44.928407 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760e484-f0d2-4520-beae-37ae0c85688f None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:44.930734 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 474/953] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:45:44.936806 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b7c37-9f27-4d50-aad3-e38edd0d5129 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:44.937138 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b7c37-9f27-4d50-aad3-e38edd0d5129 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:44.937423 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b7c37-9f27-4d50-aad3-e38edd0d5129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:44.958594 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-897b7c37-9f27-4d50-aad3-e38edd0d5129 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:44.959999 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-897b7c37-9f27-4d50-aad3-e38edd0d5129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:45.041454 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 475/954] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 06 14:45:45.046113 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d59478-842e-44cb-8c3c-8a84f9decacd None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:45.064478 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b7c37-9f27-4d50-aad3-e38edd0d5129 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:45.085386 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbef04b6-c0d6-4f6b-832e-71d3a26d443a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:45.086409 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef04b6-c0d6-4f6b-832e-71d3a26d443a None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:45.086927 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef04b6-c0d6-4f6b-832e-71d3a26d443a None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:45.087902 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef04b6-c0d6-4f6b-832e-71d3a26d443a None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:45.089205 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbef04b6-c0d6-4f6b-832e-71d3a26d443a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:45.089906 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbef04b6-c0d6-4f6b-832e-71d3a26d443a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:45.092155 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbef04b6-c0d6-4f6b-832e-71d3a26d443a None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:45.100002 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 480/955] 10.222.0.23 () {56 vars in 1122 bytes} [Fri May 6 14:45:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:45.106218 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbef04b6-c0d6-4f6b-832e-71d3a26d443a None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:45.180031 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b7c37-9f27-4d50-aad3-e38edd0d5129 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:45.215942 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60102f3-3f37-4da0-a14b-b799b09b72f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:45.216789 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60102f3-3f37-4da0-a14b-b799b09b72f6 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:45.217069 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60102f3-3f37-4da0-a14b-b799b09b72f6 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:45.217336 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60102f3-3f37-4da0-a14b-b799b09b72f6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:45.217978 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60102f3-3f37-4da0-a14b-b799b09b72f6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:45.219333 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60102f3-3f37-4da0-a14b-b799b09b72f6 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:45.235665 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-e60102f3-3f37-4da0-a14b-b799b09b72f6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 06 14:45:45.235665 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:45.235665 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 06 14:45:45.235665 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 06 14:45:45.235665 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 06 14:45:45.235665 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 06 14:45:45.235665 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 06 14:45:45.235665 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:45.235665 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 06 14:45:45.235665 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:45.235665 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 06 14:45:45.235665 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:45.235665 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:45.235665 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:45.235665 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:45.235665 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:45.235665 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:45.236741 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:45.236741 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:45.236741 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:45.236741 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:45.236741 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:45.236741 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:45.236741 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 06 14:45:45.236741 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(service_id) May 06 14:45:45.236741 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 06 14:45:45.236741 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 06 14:45:45.236741 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:45.236741 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:45.236741 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:45.236741 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:45.236741 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:45.236741 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:45.237635 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:45.237635 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:45.237635 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:45.237635 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:45.237635 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:45.237635 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:45.237635 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:45.237635 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:45.237635 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:45.237635 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:45.237635 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 06 14:45:45.237635 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:45.237635 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 06 14:45:45.237635 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:45.237635 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 476/956] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:45:45 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:45.238468 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbef04b6-c0d6-4f6b-832e-71d3a26d443a None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:45.246385 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60102f3-3f37-4da0-a14b-b799b09b72f6 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:45.279916 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c3b6a-96a6-48c8-9d26-8be941225e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:45.281074 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c3b6a-96a6-48c8-9d26-8be941225e4d None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:45.281451 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c3b6a-96a6-48c8-9d26-8be941225e4d None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:45.281827 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c3b6a-96a6-48c8-9d26-8be941225e4d None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:45.282667 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43c3b6a-96a6-48c8-9d26-8be941225e4d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:45.283086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43c3b6a-96a6-48c8-9d26-8be941225e4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:45.284524 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43c3b6a-96a6-48c8-9d26-8be941225e4d None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:45.295814 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 481/957] 10.222.0.23 () {56 vars in 1122 bytes} [Fri May 6 14:45:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:45.297093 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c3b6a-96a6-48c8-9d26-8be941225e4d None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:45.374333 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60102f3-3f37-4da0-a14b-b799b09b72f6 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:45.404146 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c3b6a-96a6-48c8-9d26-8be941225e4d None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:45.417285 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d5341b-4e3f-472b-ae95-86b2510c672a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:45.418000 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5341b-4e3f-472b-ae95-86b2510c672a None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:45.418282 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5341b-4e3f-472b-ae95-86b2510c672a None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:45.418554 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5341b-4e3f-472b-ae95-86b2510c672a None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:45.419252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d5341b-4e3f-472b-ae95-86b2510c672a None admin] RBAC: Adding query filter params (name=compute) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:45.419554 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d5341b-4e3f-472b-ae95-86b2510c672a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:45.420951 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d5341b-4e3f-472b-ae95-86b2510c672a None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:45.426841 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 477/958] 10.222.0.23 () {56 vars in 1112 bytes} [Fri May 6 14:45:45 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:45.431219 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d5341b-4e3f-472b-ae95-86b2510c672a None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:45.442128 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f94f50-ad1e-41eb-8312-c2e8cf6b808b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:45.442804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f94f50-ad1e-41eb-8312-c2e8cf6b808b None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:45.443089 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f94f50-ad1e-41eb-8312-c2e8cf6b808b None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:45.443367 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f94f50-ad1e-41eb-8312-c2e8cf6b808b None admin] PATH_INFO: `/v3/endpoints` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:45.444477 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f94f50-ad1e-41eb-8312-c2e8cf6b808b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:45.446485 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f94f50-ad1e-41eb-8312-c2e8cf6b808b None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:45.470740 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 482/959] 10.222.0.23 () {60 vars in 1144 bytes} [Fri May 6 14:45:45 2022] POST /identity/v3/endpoints => generated 349 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:45:45.552493 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d5341b-4e3f-472b-ae95-86b2510c672a None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:45.599320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b665e487-e70f-4b1d-b730-a3ad43877607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:45.599976 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665e487-e70f-4b1d-b730-a3ad43877607 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:45.600312 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665e487-e70f-4b1d-b730-a3ad43877607 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:45.600590 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665e487-e70f-4b1d-b730-a3ad43877607 None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:45.601327 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b665e487-e70f-4b1d-b730-a3ad43877607 None admin] RBAC: Adding query filter params (type=compute) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:45.601686 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b665e487-e70f-4b1d-b730-a3ad43877607 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:45.603142 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b665e487-e70f-4b1d-b730-a3ad43877607 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:45.612964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 478/960] 10.222.0.23 () {56 vars in 1112 bytes} [Fri May 6 14:45:45 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:46.821134 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0559e-a6c0-46f2-8e58-70cbceddee31 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:46.821134 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0559e-a6c0-46f2-8e58-70cbceddee31 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:46.821134 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0559e-a6c0-46f2-8e58-70cbceddee31 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:46.822358 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 483/961] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:45:46.828144 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9441d-f86e-4f32-8915-d7060594bb4c None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:46.829009 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9441d-f86e-4f32-8915-d7060594bb4c None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:46.829502 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9441d-f86e-4f32-8915-d7060594bb4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:46.846686 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-e9b9441d-f86e-4f32-8915-d7060594bb4c None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:46.848371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-e9b9441d-f86e-4f32-8915-d7060594bb4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:46.917153 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 479/962] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 06 14:45:46.942191 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0559e-a6c0-46f2-8e58-70cbceddee31 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:47.047878 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0559e-a6c0-46f2-8e58-70cbceddee31 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:47.081142 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287fb53b-4d21-4688-b762-bf0d68b829fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:47.082190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287fb53b-4d21-4688-b762-bf0d68b829fc None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:47.082683 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287fb53b-4d21-4688-b762-bf0d68b829fc None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:47.083158 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287fb53b-4d21-4688-b762-bf0d68b829fc None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:47.084139 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287fb53b-4d21-4688-b762-bf0d68b829fc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:47.085798 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287fb53b-4d21-4688-b762-bf0d68b829fc None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:47.099738 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.sql.core [None req-287fb53b-4d21-4688-b762-bf0d68b829fc None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 06 14:45:47.099738 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 06 14:45:47.099738 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [parameters: {'user_id': '474211258cb84c93a436e1a3ccd45548', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 06 14:45:47.099738 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=79888) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 06 14:45:47.108048 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-287fb53b-4d21-4688-b762-bf0d68b829fc None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 06 14:45:47.108048 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:47.108048 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 06 14:45:47.108048 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self.dialect.do_execute( May 06 14:45:47.108048 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 06 14:45:47.108048 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 06 14:45:47.108048 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 06 14:45:47.108048 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application result = self._query(query) May 06 14:45:47.108048 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 06 14:45:47.108048 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application conn.query(q) May 06 14:45:47.108048 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 06 14:45:47.108048 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 06 14:45:47.108048 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 06 14:45:47.108048 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application result.read() May 06 14:45:47.108048 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 06 14:45:47.108986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 06 14:45:47.108986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 06 14:45:47.108986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application packet.raise_for_error() May 06 14:45:47.108986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 06 14:45:47.108986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 06 14:45:47.108986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 06 14:45:47.108986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 06 14:45:47.108986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 06 14:45:47.108986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:47.108986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 06 14:45:47.108986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:47.108986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:47.108986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 06 14:45:47.108986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 06 14:45:47.108986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 06 14:45:47.108986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 06 14:45:47.108986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 06 14:45:47.108986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application next(self.gen) May 06 14:45:47.109959 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 06 14:45:47.109959 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application yield resource May 06 14:45:47.109959 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 06 14:45:47.109959 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application next(self.gen) May 06 14:45:47.109959 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 06 14:45:47.109959 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self.session.rollback() May 06 14:45:47.109959 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 06 14:45:47.109959 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self.force_reraise() May 06 14:45:47.109959 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 06 14:45:47.109959 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise self.value May 06 14:45:47.109959 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 06 14:45:47.109959 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 06 14:45:47.109959 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 06 14:45:47.109959 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application session.commit() May 06 14:45:47.109959 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 06 14:45:47.109959 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 06 14:45:47.109959 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 06 14:45:47.110870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._prepare_impl() May 06 14:45:47.110870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 06 14:45:47.110870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self.session.flush() May 06 14:45:47.110870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 06 14:45:47.110870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._flush(objects) May 06 14:45:47.110870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 06 14:45:47.110870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 06 14:45:47.110870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 06 14:45:47.110870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application compat.raise_( May 06 14:45:47.110870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 06 14:45:47.110870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception May 06 14:45:47.110870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 06 14:45:47.110870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application flush_context.execute() May 06 14:45:47.110870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 06 14:45:47.110870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rec.execute(self) May 06 14:45:47.110870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 06 14:45:47.110870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 06 14:45:47.112592 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 06 14:45:47.112592 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application _emit_insert_statements( May 06 14:45:47.112592 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 06 14:45:47.112592 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application result = connection._execute_20( May 06 14:45:47.112592 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 06 14:45:47.112592 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 06 14:45:47.112592 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 06 14:45:47.112592 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 06 14:45:47.112592 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 06 14:45:47.112592 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ret = self._execute_context( May 06 14:45:47.112592 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 06 14:45:47.112592 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 06 14:45:47.112592 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 06 14:45:47.112592 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 06 14:45:47.112592 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 06 14:45:47.112592 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception May 06 14:45:47.114535 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 06 14:45:47.114535 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self.dialect.do_execute( May 06 14:45:47.114535 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 06 14:45:47.114535 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 06 14:45:47.114535 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 06 14:45:47.114535 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application result = self._query(query) May 06 14:45:47.114535 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 06 14:45:47.114535 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application conn.query(q) May 06 14:45:47.114535 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 06 14:45:47.114535 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 06 14:45:47.114535 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 06 14:45:47.114535 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application result.read() May 06 14:45:47.114535 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 06 14:45:47.114535 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 06 14:45:47.114535 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 06 14:45:47.114535 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application packet.raise_for_error() May 06 14:45:47.115393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 06 14:45:47.115393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 06 14:45:47.115393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 06 14:45:47.115393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 06 14:45:47.115393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 06 14:45:47.115393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 06 14:45:47.115393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [parameters: {'user_id': '474211258cb84c93a436e1a3ccd45548', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 06 14:45:47.115393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 06 14:45:47.115393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:47.115393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 06 14:45:47.115393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:47.115393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:47.115393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:47.115393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:47.115393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:47.117247 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:47.117247 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:47.117247 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:47.117247 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:47.117247 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:47.117247 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:47.117247 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:47.117247 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 06 14:45:47.117247 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 06 14:45:47.117247 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:47.117247 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:47.117247 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 06 14:45:47.117247 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:47.117247 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 06 14:45:47.117247 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:47.117247 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 06 14:45:47.117247 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 06 14:45:47.119401 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 06 14:45:47.119401 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 06 14:45:47.119401 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 06 14:45:47.119401 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 06 14:45:47.119401 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 06 14:45:47.119401 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:47.121969 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 484/963] 10.222.0.23 () {60 vars in 1131 bytes} [Fri May 6 14:45:46 2022] POST /identity/v3/users => generated 160 bytes in 180 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:47.129751 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9441d-f86e-4f32-8915-d7060594bb4c None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:47.183192 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0d02e-8a7e-4f46-a430-d9a8b10fd504 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:47.183499 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0d02e-8a7e-4f46-a430-d9a8b10fd504 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:47.183784 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0d02e-8a7e-4f46-a430-d9a8b10fd504 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:47.190801 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 485/964] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:47 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:45:47.197001 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d089a2a-9195-42f8-aeaa-1bd81161dd41 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:47.197333 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d089a2a-9195-42f8-aeaa-1bd81161dd41 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:47.197616 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d089a2a-9195-42f8-aeaa-1bd81161dd41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:47.219468 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-5d089a2a-9195-42f8-aeaa-1bd81161dd41 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:47.221044 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-5d089a2a-9195-42f8-aeaa-1bd81161dd41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:47.245403 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9441d-f86e-4f32-8915-d7060594bb4c None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:47.281689 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8300a4c-1964-4ee5-9d94-86462aff3bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:47.282880 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8300a4c-1964-4ee5-9d94-86462aff3bd0 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:47.283361 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8300a4c-1964-4ee5-9d94-86462aff3bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:47.283829 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8300a4c-1964-4ee5-9d94-86462aff3bd0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:47.292092 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8300a4c-1964-4ee5-9d94-86462aff3bd0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:47.293904 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8300a4c-1964-4ee5-9d94-86462aff3bd0 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:47.310338 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-e8300a4c-1964-4ee5-9d94-86462aff3bd0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 06 14:45:47.310338 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:47.310338 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:47.310338 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:47.310338 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:47.310338 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:47.310338 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:47.310338 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:47.310338 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:47.310338 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:47.310338 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:47.310338 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:47.310338 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 06 14:45:47.310338 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_user(user_id) May 06 14:45:47.310338 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 06 14:45:47.310338 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 06 14:45:47.311308 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:47.311308 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:47.311308 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 06 14:45:47.311308 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:47.311308 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 06 14:45:47.311308 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:47.311308 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:47.311308 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:47.311308 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:47.311308 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:47.311308 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:47.311308 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:47.311308 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:47.311308 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:47.311308 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:47.311308 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:47.311308 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:47.311308 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:47.312249 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:47.312249 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:47.312249 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 06 14:45:47.312249 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 06 14:45:47.312249 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 06 14:45:47.312249 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 06 14:45:47.312249 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 06 14:45:47.312249 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 06 14:45:47.312249 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 06 14:45:47.312249 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:47.313341 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 486/965] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:45:47.315217 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 480/966] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:45:47 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:47.320964 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d089a2a-9195-42f8-aeaa-1bd81161dd41 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:47.334309 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8300a4c-1964-4ee5-9d94-86462aff3bd0 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:47.427369 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d089a2a-9195-42f8-aeaa-1bd81161dd41 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:47.437686 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8300a4c-1964-4ee5-9d94-86462aff3bd0 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:47.462749 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80021daf-039d-4f4e-9f78-0834c8002a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:47.463544 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80021daf-039d-4f4e-9f78-0834c8002a9e None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:47.463950 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80021daf-039d-4f4e-9f78-0834c8002a9e None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:47.464143 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80021daf-039d-4f4e-9f78-0834c8002a9e None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:47.464934 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80021daf-039d-4f4e-9f78-0834c8002a9e None admin] RBAC: Adding query filter params (name=cinder) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:47.465244 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80021daf-039d-4f4e-9f78-0834c8002a9e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:47.466998 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80021daf-039d-4f4e-9f78-0834c8002a9e None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:47.474861 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79131d61-0419-41b8-89c8-b34f818205db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:47.476008 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79131d61-0419-41b8-89c8-b34f818205db None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:47.476512 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79131d61-0419-41b8-89c8-b34f818205db None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:47.477026 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79131d61-0419-41b8-89c8-b34f818205db None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:47.488729 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79131d61-0419-41b8-89c8-b34f818205db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:45:47.497475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-79131d61-0419-41b8-89c8-b34f818205db None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 06 14:45:47.497475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:47.497475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:47.497475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:47.497475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:47.497475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:47.497475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:47.497475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:47.497475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:47.497475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:47.497475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:47.497475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:47.497475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:45:47.497475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:45:47.497475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:45:47.497475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:45:47.498734 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:45:47.498734 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:45:47.498734 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:45:47.498734 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:45:47.498734 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:47.498734 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:47.498734 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:47.498734 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:47.498734 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:47.498734 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:47.498734 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:47.498734 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:47.498734 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:47.498734 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:47.498734 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:47.498734 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:47.499617 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:47.499617 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:47.499617 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:47.499617 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:47.499617 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:45:47.499617 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:45:47.499617 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:45:47.499617 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:45:47.499617 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:45:47.499617 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:45:47.499617 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 06 14:45:47.499617 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:47.508102 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 481/967] 10.222.0.23 () {56 vars in 1120 bytes} [Fri May 6 14:45:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:47.509492 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79131d61-0419-41b8-89c8-b34f818205db None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:47.513022 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 487/968] 10.222.0.23 () {56 vars in 1101 bytes} [Fri May 6 14:45:47 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:47.623240 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79131d61-0419-41b8-89c8-b34f818205db None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:47.662104 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2595b9b-dd87-46b5-8ca7-cb3146e12b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:47.663295 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2595b9b-dd87-46b5-8ca7-cb3146e12b8a None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:47.663785 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2595b9b-dd87-46b5-8ca7-cb3146e12b8a None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:47.664300 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2595b9b-dd87-46b5-8ca7-cb3146e12b8a None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:47.665324 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2595b9b-dd87-46b5-8ca7-cb3146e12b8a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:47.665863 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2595b9b-dd87-46b5-8ca7-cb3146e12b8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:47.667458 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2595b9b-dd87-46b5-8ca7-cb3146e12b8a None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:47.678205 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 482/969] 10.222.0.23 () {56 vars in 1115 bytes} [Fri May 6 14:45:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:47.684511 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80021daf-039d-4f4e-9f78-0834c8002a9e None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:47.796054 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80021daf-039d-4f4e-9f78-0834c8002a9e None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:47.831058 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745a54b-57b7-4144-bc6a-7f77f347bf0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:47.832748 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745a54b-57b7-4144-bc6a-7f77f347bf0c None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:47.833046 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745a54b-57b7-4144-bc6a-7f77f347bf0c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:47.833330 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745a54b-57b7-4144-bc6a-7f77f347bf0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:47.843693 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2745a54b-57b7-4144-bc6a-7f77f347bf0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:47.845176 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2745a54b-57b7-4144-bc6a-7f77f347bf0c None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:47.863981 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-2745a54b-57b7-4144-bc6a-7f77f347bf0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:47.863981 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:47.863981 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:47.863981 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:47.863981 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:47.863981 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:47.863981 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:47.863981 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:47.863981 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:47.863981 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:47.863981 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:47.863981 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:47.863981 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:47.863981 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:47.863981 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:47.865076 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:47.865076 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:47.865076 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:47.865076 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:47.865076 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:47.865076 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:47.865076 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:47.865076 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:47.865076 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:47.865076 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:47.865076 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:47.865076 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:47.865076 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:47.865076 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:47.865076 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:47.865076 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:47.865076 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:47.865962 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:47.865962 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:47.865962 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:47.865962 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:47.865962 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 488/970] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:47.872014 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2595b9b-dd87-46b5-8ca7-cb3146e12b8a None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:47.885519 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c0a66-3078-4555-8115-ee5a36ba59ee None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:47.885794 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c0a66-3078-4555-8115-ee5a36ba59ee None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:47.886015 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c0a66-3078-4555-8115-ee5a36ba59ee None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:47.887368 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 489/971] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:45:47.892333 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ec42e-be6f-4287-b66e-1f8ffdf1727d None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:47.892651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ec42e-be6f-4287-b66e-1f8ffdf1727d None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:47.892935 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ec42e-be6f-4287-b66e-1f8ffdf1727d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:47.910389 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-df9ec42e-be6f-4287-b66e-1f8ffdf1727d None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:47.911717 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-df9ec42e-be6f-4287-b66e-1f8ffdf1727d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:47.971211 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2595b9b-dd87-46b5-8ca7-cb3146e12b8a None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:47.977813 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 490/972] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 06 14:45:47.979131 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655001b7-bc21-4f1b-9df6-cebc6e5b1168 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:47.979419 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655001b7-bc21-4f1b-9df6-cebc6e5b1168 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:47.979694 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655001b7-bc21-4f1b-9df6-cebc6e5b1168 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:47.984632 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 491/973] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:45:47.986318 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc96d6-2892-48ff-a030-f0eb4cc094f3 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:47.986675 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc96d6-2892-48ff-a030-f0eb4cc094f3 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:47.986954 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc96d6-2892-48ff-a030-f0eb4cc094f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:48.005274 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-34fc96d6-2892-48ff-a030-f0eb4cc094f3 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:48.006441 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-34fc96d6-2892-48ff-a030-f0eb4cc094f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:48.006779 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5270448-c660-49ad-9403-2eb48d897012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:48.007439 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5270448-c660-49ad-9403-2eb48d897012 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:48.007741 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5270448-c660-49ad-9403-2eb48d897012 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:48.007998 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5270448-c660-49ad-9403-2eb48d897012 None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:48.008861 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5270448-c660-49ad-9403-2eb48d897012 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:48.009171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5270448-c660-49ad-9403-2eb48d897012 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:48.010530 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5270448-c660-49ad-9403-2eb48d897012 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:48.022816 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 483/974] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:48.023620 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5270448-c660-49ad-9403-2eb48d897012 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:48.073712 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 492/975] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 06 14:45:48.074885 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc96d6-2892-48ff-a030-f0eb4cc094f3 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:48.124580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5270448-c660-49ad-9403-2eb48d897012 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:48.160112 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66477775-4c91-404c-a3d1-58d3c57458c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:48.160954 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66477775-4c91-404c-a3d1-58d3c57458c1 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:48.161234 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66477775-4c91-404c-a3d1-58d3c57458c1 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:48.161509 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66477775-4c91-404c-a3d1-58d3c57458c1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:48.162153 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66477775-4c91-404c-a3d1-58d3c57458c1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:48.163516 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66477775-4c91-404c-a3d1-58d3c57458c1 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:48.171276 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc96d6-2892-48ff-a030-f0eb4cc094f3 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:48.175251 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-66477775-4c91-404c-a3d1-58d3c57458c1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 06 14:45:48.175251 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:48.175251 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 06 14:45:48.175251 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 06 14:45:48.175251 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 06 14:45:48.175251 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 06 14:45:48.175251 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 06 14:45:48.175251 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:48.175251 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 06 14:45:48.175251 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:48.175251 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 06 14:45:48.175251 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:48.175251 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:48.175251 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:48.175251 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:48.175251 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:48.175251 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:48.177546 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:48.177546 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:48.177546 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:48.177546 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:48.177546 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:48.177546 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:48.177546 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 06 14:45:48.177546 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(service_id) May 06 14:45:48.177546 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 06 14:45:48.177546 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 06 14:45:48.177546 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:48.177546 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:48.177546 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:48.177546 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:48.177546 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:48.177546 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:48.178385 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:48.178385 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:48.178385 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:48.178385 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:48.178385 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:48.178385 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:48.178385 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:48.178385 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:48.178385 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:48.178385 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:48.178385 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 06 14:45:48.178385 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:48.178385 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 06 14:45:48.178385 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:48.178385 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 484/976] 10.222.0.23 () {56 vars in 1126 bytes} [Fri May 6 14:45:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:48.179192 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66477775-4c91-404c-a3d1-58d3c57458c1 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:48.204722 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2deb47b-a183-40a0-b4b8-4756d078e863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:48.205535 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2deb47b-a183-40a0-b4b8-4756d078e863 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:48.205818 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2deb47b-a183-40a0-b4b8-4756d078e863 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:48.206089 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2deb47b-a183-40a0-b4b8-4756d078e863 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:48.213059 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2deb47b-a183-40a0-b4b8-4756d078e863 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:48.214493 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2deb47b-a183-40a0-b4b8-4756d078e863 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:48.229127 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-c2deb47b-a183-40a0-b4b8-4756d078e863 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 06 14:45:48.229127 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:48.229127 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:48.229127 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:48.229127 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:48.229127 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:48.229127 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:48.229127 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:48.229127 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:48.229127 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:48.229127 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:48.229127 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:48.229127 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 06 14:45:48.229127 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_user(user_id) May 06 14:45:48.229127 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 06 14:45:48.229127 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 06 14:45:48.230267 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:48.230267 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:48.230267 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 06 14:45:48.230267 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:48.230267 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 06 14:45:48.230267 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:48.230267 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:48.230267 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:48.230267 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:48.230267 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:48.230267 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:48.230267 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:48.230267 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:48.230267 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:48.230267 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:48.230267 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:48.230267 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:48.230267 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:48.231418 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:48.231418 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:48.231418 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 06 14:45:48.231418 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 06 14:45:48.231418 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 06 14:45:48.231418 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 06 14:45:48.231418 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 06 14:45:48.231418 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 06 14:45:48.231418 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 06 14:45:48.231418 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:48.231418 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 493/977] 10.222.0.23 () {56 vars in 1117 bytes} [Fri May 6 14:45:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:48.231418 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2deb47b-a183-40a0-b4b8-4756d078e863 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:48.279989 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66477775-4c91-404c-a3d1-58d3c57458c1 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:48.312755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061e222-cd94-49d0-9df3-68219aa1a593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:48.313406 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061e222-cd94-49d0-9df3-68219aa1a593 None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:48.313686 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061e222-cd94-49d0-9df3-68219aa1a593 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:48.313960 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061e222-cd94-49d0-9df3-68219aa1a593 None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:48.314658 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4061e222-cd94-49d0-9df3-68219aa1a593 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:48.316006 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4061e222-cd94-49d0-9df3-68219aa1a593 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:48.326382 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2deb47b-a183-40a0-b4b8-4756d078e863 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:48.331027 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 485/978] 10.222.0.23 () {60 vars in 1141 bytes} [Fri May 6 14:45:48 2022] POST /identity/v3/services => generated 242 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:45:48.331821 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061e222-cd94-49d0-9df3-68219aa1a593 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:48.372572 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d892778-f141-42c5-a1df-377ebffa5be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:48.373284 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d892778-f141-42c5-a1df-377ebffa5be9 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:48.373565 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d892778-f141-42c5-a1df-377ebffa5be9 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:48.373836 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d892778-f141-42c5-a1df-377ebffa5be9 None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:48.374544 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d892778-f141-42c5-a1df-377ebffa5be9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:48.374847 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d892778-f141-42c5-a1df-377ebffa5be9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:48.376238 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d892778-f141-42c5-a1df-377ebffa5be9 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:48.382615 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 494/979] 10.222.0.23 () {56 vars in 1122 bytes} [Fri May 6 14:45:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:48.388390 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d892778-f141-42c5-a1df-377ebffa5be9 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:48.454461 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061e222-cd94-49d0-9df3-68219aa1a593 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:48.489213 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8979d765-8e2d-44bc-ba15-f91a9c259725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:48.489834 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979d765-8e2d-44bc-ba15-f91a9c259725 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:48.490187 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979d765-8e2d-44bc-ba15-f91a9c259725 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:48.490388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979d765-8e2d-44bc-ba15-f91a9c259725 None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:48.491232 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8979d765-8e2d-44bc-ba15-f91a9c259725 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:48.491542 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8979d765-8e2d-44bc-ba15-f91a9c259725 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:48.493092 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8979d765-8e2d-44bc-ba15-f91a9c259725 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:48.499543 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d892778-f141-42c5-a1df-377ebffa5be9 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:48.517240 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 486/980] 10.222.0.23 () {56 vars in 1149 bytes} [Fri May 6 14:45:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:48.523042 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8979d765-8e2d-44bc-ba15-f91a9c259725 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:48.536662 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d409dfe-88cc-4e95-8dec-d50fdea05be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:48.537346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d409dfe-88cc-4e95-8dec-d50fdea05be0 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:48.537638 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d409dfe-88cc-4e95-8dec-d50fdea05be0 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:48.538021 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d409dfe-88cc-4e95-8dec-d50fdea05be0 None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:48.539421 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d409dfe-88cc-4e95-8dec-d50fdea05be0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:48.539421 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d409dfe-88cc-4e95-8dec-d50fdea05be0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:48.540296 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d409dfe-88cc-4e95-8dec-d50fdea05be0 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:48.545807 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 495/981] 10.222.0.23 () {56 vars in 1122 bytes} [Fri May 6 14:45:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:48.551896 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d409dfe-88cc-4e95-8dec-d50fdea05be0 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:48.623673 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8979d765-8e2d-44bc-ba15-f91a9c259725 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:48.652579 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d409dfe-88cc-4e95-8dec-d50fdea05be0 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:48.657223 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1870b22d-9ea2-4bed-bdd9-d5adf16d0ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:48.657871 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1870b22d-9ea2-4bed-bdd9-d5adf16d0ddf None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:48.658152 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1870b22d-9ea2-4bed-bdd9-d5adf16d0ddf None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:48.658430 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1870b22d-9ea2-4bed-bdd9-d5adf16d0ddf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:48.687943 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82229753-cd83-424e-8533-bca4383fd850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:48.689011 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82229753-cd83-424e-8533-bca4383fd850 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:48.689359 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82229753-cd83-424e-8533-bca4383fd850 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:48.689638 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82229753-cd83-424e-8533-bca4383fd850 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:48.690427 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82229753-cd83-424e-8533-bca4383fd850 None admin] RBAC: Adding query filter params (interface=admin, service_id=4af42964328248c7bea104867d364c8d, region_id=RegionOne) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:48.690746 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82229753-cd83-424e-8533-bca4383fd850 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:48.692399 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82229753-cd83-424e-8533-bca4383fd850 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:48.697728 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 496/982] 10.222.0.23 () {56 vars in 1249 bytes} [Fri May 6 14:45:48 2022] GET /identity/v3/endpoints?service_id=4af42964328248c7bea104867d364c8d&interface=admin®ion_id=RegionOne => generated 193 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:48.702975 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82229753-cd83-424e-8533-bca4383fd850 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:48.728511 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1870b22d-9ea2-4bed-bdd9-d5adf16d0ddf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:48.730263 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1870b22d-9ea2-4bed-bdd9-d5adf16d0ddf None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:48.799996 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82229753-cd83-424e-8533-bca4383fd850 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:48.830115 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 487/983] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:45:48 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 06 14:45:48.833430 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19911c3c-dd6d-4bfd-a52c-a84ec8d9755f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:48.834098 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19911c3c-dd6d-4bfd-a52c-a84ec8d9755f None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:48.834379 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19911c3c-dd6d-4bfd-a52c-a84ec8d9755f None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:48.834655 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19911c3c-dd6d-4bfd-a52c-a84ec8d9755f None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:48.835365 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19911c3c-dd6d-4bfd-a52c-a84ec8d9755f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:48.836514 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1870b22d-9ea2-4bed-bdd9-d5adf16d0ddf None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:48.837130 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19911c3c-dd6d-4bfd-a52c-a84ec8d9755f None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:48.843904 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 497/984] 10.222.0.23 () {56 vars in 1087 bytes} [Fri May 6 14:45:48 2022] GET /identity/v3/services => generated 1727 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:45:48.950732 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1870b22d-9ea2-4bed-bdd9-d5adf16d0ddf None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:48.987081 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d8efd-df20-4f55-8ba2-519703e3ecf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:48.987884 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d8efd-df20-4f55-8ba2-519703e3ecf9 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:48.988276 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d8efd-df20-4f55-8ba2-519703e3ecf9 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:48.988553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d8efd-df20-4f55-8ba2-519703e3ecf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:48.999383 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900d8efd-df20-4f55-8ba2-519703e3ecf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:49.000886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900d8efd-df20-4f55-8ba2-519703e3ecf9 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:49.020369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-900d8efd-df20-4f55-8ba2-519703e3ecf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:49.020369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:49.020369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:49.020369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:49.020369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:49.020369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:49.020369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:49.020369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:49.020369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:49.020369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:49.020369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:49.020369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:49.020369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:49.020369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:49.020369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:49.021388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:49.021388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:49.021388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:49.021388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:49.021388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:49.021388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:49.021388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:49.021388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:49.021388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:49.021388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:49.021388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:49.021388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:49.021388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:49.021388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:49.021388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:49.021388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:49.021388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:49.022396 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:49.022396 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:49.022396 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:49.022396 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:49.022396 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 488/985] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:45:49.028262 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19911c3c-dd6d-4bfd-a52c-a84ec8d9755f None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:49.128425 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19911c3c-dd6d-4bfd-a52c-a84ec8d9755f None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:49.170685 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834b5483-2e61-4be1-b7aa-5dd030994e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:49.171833 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b5483-2e61-4be1-b7aa-5dd030994e45 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:49.172339 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b5483-2e61-4be1-b7aa-5dd030994e45 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:49.172884 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b5483-2e61-4be1-b7aa-5dd030994e45 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:49.174218 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834b5483-2e61-4be1-b7aa-5dd030994e45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:49.174776 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834b5483-2e61-4be1-b7aa-5dd030994e45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:49.177910 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834b5483-2e61-4be1-b7aa-5dd030994e45 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:49.197124 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 498/986] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:49.198508 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d8efd-df20-4f55-8ba2-519703e3ecf9 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:49.302656 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d8efd-df20-4f55-8ba2-519703e3ecf9 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:49.334784 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7507d4-fa98-4e6d-91c8-52de5c9202c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:49.335575 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7507d4-fa98-4e6d-91c8-52de5c9202c4 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:49.335856 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7507d4-fa98-4e6d-91c8-52de5c9202c4 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:49.336127 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7507d4-fa98-4e6d-91c8-52de5c9202c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:49.342629 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7507d4-fa98-4e6d-91c8-52de5c9202c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:49.344051 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7507d4-fa98-4e6d-91c8-52de5c9202c4 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:49.358951 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-ee7507d4-fa98-4e6d-91c8-52de5c9202c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:49.358951 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:49.358951 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:49.358951 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:49.358951 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:49.358951 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:49.358951 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:49.358951 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:49.358951 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:49.358951 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:49.358951 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:49.358951 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:49.358951 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 06 14:45:49.358951 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(project_id) May 06 14:45:49.358951 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 06 14:45:49.359927 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 06 14:45:49.359927 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:49.359927 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:49.359927 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:49.359927 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:49.359927 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:49.359927 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:49.359927 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:49.359927 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:49.359927 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:49.359927 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:49.359927 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:49.359927 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:49.359927 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:49.359927 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:49.359927 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:49.359927 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:49.360895 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 06 14:45:49.360895 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 06 14:45:49.360895 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 06 14:45:49.360895 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 06 14:45:49.360895 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 06 14:45:49.360895 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 06 14:45:49.360895 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:49.360895 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:49.361916 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 489/987] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:45:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:45:49.366686 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834b5483-2e61-4be1-b7aa-5dd030994e45 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:49.465003 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834b5483-2e61-4be1-b7aa-5dd030994e45 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:49.500584 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b811b5-c70c-4bdb-a9e0-414c6fc16ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:49.501280 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b811b5-c70c-4bdb-a9e0-414c6fc16ff9 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:49.501567 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b811b5-c70c-4bdb-a9e0-414c6fc16ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:49.501848 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b811b5-c70c-4bdb-a9e0-414c6fc16ff9 None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:49.502630 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b811b5-c70c-4bdb-a9e0-414c6fc16ff9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:49.502941 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b811b5-c70c-4bdb-a9e0-414c6fc16ff9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:49.504556 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b811b5-c70c-4bdb-a9e0-414c6fc16ff9 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:49.516502 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 499/988] 10.222.0.23 () {56 vars in 1148 bytes} [Fri May 6 14:45:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:49.523069 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7507d4-fa98-4e6d-91c8-52de5c9202c4 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:49.623426 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7507d4-fa98-4e6d-91c8-52de5c9202c4 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:49.657095 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95428118-a345-4c0b-9c6a-548996ff8733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:49.657747 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95428118-a345-4c0b-9c6a-548996ff8733 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:49.658027 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95428118-a345-4c0b-9c6a-548996ff8733 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:49.658302 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95428118-a345-4c0b-9c6a-548996ff8733 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:49.659142 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95428118-a345-4c0b-9c6a-548996ff8733 None admin] RBAC: Adding query filter params (role.id=5213da7472024e87a41cecfaa37abce3, scope.project.id=d5398feee6eb490cb49bb261448a1a9b, user.id=a7dc1bc53c59423e934f7831ea022c73) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:49.659452 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95428118-a345-4c0b-9c6a-548996ff8733 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:49.661975 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95428118-a345-4c0b-9c6a-548996ff8733 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:49.667799 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 490/989] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:45:49 2022] GET /identity/v3/role_assignments?user.id=a7dc1bc53c59423e934f7831ea022c73&scope.project.id=d5398feee6eb490cb49bb261448a1a9b&role.id=5213da7472024e87a41cecfaa37abce3 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:50.624300 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc0e08-ab94-4fe8-b2c1-d46fad7561aa None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:50.624866 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc0e08-ab94-4fe8-b2c1-d46fad7561aa None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:50.624866 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc0e08-ab94-4fe8-b2c1-d46fad7561aa None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:50.626392 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 500/990] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:45:50.632173 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f92d5d-4802-413c-ac04-ba82d5fb9032 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:50.632561 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f92d5d-4802-413c-ac04-ba82d5fb9032 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:50.632764 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f92d5d-4802-413c-ac04-ba82d5fb9032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:50.649670 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-89f92d5d-4802-413c-ac04-ba82d5fb9032 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:50.650845 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-89f92d5d-4802-413c-ac04-ba82d5fb9032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:50.718180 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 491/991] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:50 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:45:50.749010 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc0e08-ab94-4fe8-b2c1-d46fad7561aa None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:50.862615 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc0e08-ab94-4fe8-b2c1-d46fad7561aa None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:50.898112 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3be7ac-a7db-452c-a0e1-d5cb342e2f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:50.899266 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3be7ac-a7db-452c-a0e1-d5cb342e2f5a None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:50.899775 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3be7ac-a7db-452c-a0e1-d5cb342e2f5a None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:50.900254 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3be7ac-a7db-452c-a0e1-d5cb342e2f5a None admin] PATH_INFO: `/v3/services/compute` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:50.901158 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3be7ac-a7db-452c-a0e1-d5cb342e2f5a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:50.902727 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3be7ac-a7db-452c-a0e1-d5cb342e2f5a None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:50.916341 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-9a3be7ac-a7db-452c-a0e1-d5cb342e2f5a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 06 14:45:50.916341 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:50.916341 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 06 14:45:50.916341 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 06 14:45:50.916341 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 06 14:45:50.916341 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 06 14:45:50.916341 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 06 14:45:50.916341 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:50.916341 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 06 14:45:50.916341 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:50.916341 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 06 14:45:50.916341 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:50.916341 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:50.916341 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:50.916341 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:50.916341 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:50.916341 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:50.917418 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:50.917418 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:50.917418 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:50.917418 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:50.917418 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:50.917418 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:50.917418 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 06 14:45:50.917418 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(service_id) May 06 14:45:50.917418 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 06 14:45:50.917418 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 06 14:45:50.917418 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:50.917418 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:50.917418 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:50.917418 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:50.917418 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:50.917418 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:50.918269 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:50.918269 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:50.918269 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:50.918269 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:50.918269 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:50.918269 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:50.918269 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:50.918269 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:50.918269 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:50.918269 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:50.918269 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 06 14:45:50.918269 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:50.918269 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 06 14:45:50.918269 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:50.921382 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 501/992] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:45:50 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:50.932016 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f92d5d-4802-413c-ac04-ba82d5fb9032 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:51.037372 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f92d5d-4802-413c-ac04-ba82d5fb9032 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:51.072176 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef37672f-2d8f-4be4-94e4-d0b98239c469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:51.073275 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef37672f-2d8f-4be4-94e4-d0b98239c469 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:51.073823 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef37672f-2d8f-4be4-94e4-d0b98239c469 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:51.074324 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef37672f-2d8f-4be4-94e4-d0b98239c469 None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:51.075249 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef37672f-2d8f-4be4-94e4-d0b98239c469 None admin] RBAC: Adding query filter params (name=compute) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:51.075763 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef37672f-2d8f-4be4-94e4-d0b98239c469 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:51.077401 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef37672f-2d8f-4be4-94e4-d0b98239c469 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:51.083384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 492/993] 10.222.0.23 () {56 vars in 1112 bytes} [Fri May 6 14:45:50 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:51.088957 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3be7ac-a7db-452c-a0e1-d5cb342e2f5a None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:51.132742 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f9dec-b27b-403b-b682-275d78f2f8ef None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:51.133080 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f9dec-b27b-403b-b682-275d78f2f8ef None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:51.133315 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f9dec-b27b-403b-b682-275d78f2f8ef None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:51.134983 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 493/994] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:45:51.140444 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa684d35-9d33-488e-8f00-9e96db86e5b5 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:51.140444 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa684d35-9d33-488e-8f00-9e96db86e5b5 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:51.140815 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa684d35-9d33-488e-8f00-9e96db86e5b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:51.164574 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-aa684d35-9d33-488e-8f00-9e96db86e5b5 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:51.166128 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-aa684d35-9d33-488e-8f00-9e96db86e5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:51.208840 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3be7ac-a7db-452c-a0e1-d5cb342e2f5a None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:51.243897 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18789a04-52ff-4347-8d3c-86e70ac6f957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:51.244721 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18789a04-52ff-4347-8d3c-86e70ac6f957 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:51.245011 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18789a04-52ff-4347-8d3c-86e70ac6f957 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:51.245288 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18789a04-52ff-4347-8d3c-86e70ac6f957 None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:51.246463 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 494/995] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:51 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 06 14:45:51.246573 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18789a04-52ff-4347-8d3c-86e70ac6f957 None admin] RBAC: Adding query filter params (type=compute) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:51.246735 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18789a04-52ff-4347-8d3c-86e70ac6f957 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:51.248051 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18789a04-52ff-4347-8d3c-86e70ac6f957 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:51.257917 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 502/996] 10.222.0.23 () {56 vars in 1112 bytes} [Fri May 6 14:45:51 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:51.259075 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa684d35-9d33-488e-8f00-9e96db86e5b5 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:51.271387 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18789a04-52ff-4347-8d3c-86e70ac6f957 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:51.360230 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa684d35-9d33-488e-8f00-9e96db86e5b5 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:51.373524 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18789a04-52ff-4347-8d3c-86e70ac6f957 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:51.394083 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d8139-c81c-4be6-8fad-631618cc07fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:51.395306 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d8139-c81c-4be6-8fad-631618cc07fd None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:51.395588 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d8139-c81c-4be6-8fad-631618cc07fd None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:51.395918 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d8139-c81c-4be6-8fad-631618cc07fd None admin] PATH_INFO: `/v3/endpoints` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:51.396757 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340d8139-c81c-4be6-8fad-631618cc07fd None admin] RBAC: Adding query filter params (interface=public, service_id=63cabe3006654b77adff64d7c781048a, region_id=RegionOne) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:51.397075 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340d8139-c81c-4be6-8fad-631618cc07fd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:51.398443 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340d8139-c81c-4be6-8fad-631618cc07fd None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:51.408181 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 495/997] 10.222.0.23 () {56 vars in 1251 bytes} [Fri May 6 14:45:51 2022] GET /identity/v3/endpoints?service_id=63cabe3006654b77adff64d7c781048a&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:51.409247 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d8139-c81c-4be6-8fad-631618cc07fd None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:51.412977 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c62e5e9-fdd3-4e85-a007-bc8b5b03f0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:51.413802 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c62e5e9-fdd3-4e85-a007-bc8b5b03f0a8 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:51.414087 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c62e5e9-fdd3-4e85-a007-bc8b5b03f0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:51.414365 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c62e5e9-fdd3-4e85-a007-bc8b5b03f0a8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:51.415021 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c62e5e9-fdd3-4e85-a007-bc8b5b03f0a8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:51.416395 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c62e5e9-fdd3-4e85-a007-bc8b5b03f0a8 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:51.428733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-3c62e5e9-fdd3-4e85-a007-bc8b5b03f0a8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 06 14:45:51.428733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:51.428733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 06 14:45:51.428733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 06 14:45:51.428733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 06 14:45:51.428733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 06 14:45:51.428733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 06 14:45:51.428733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:51.428733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 06 14:45:51.428733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:51.428733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 06 14:45:51.428733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:51.428733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:51.428733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:51.428733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:51.428733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:51.428733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:51.429834 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:51.429834 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:51.429834 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:51.429834 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:51.429834 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:51.429834 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:51.429834 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 06 14:45:51.429834 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(service_id) May 06 14:45:51.429834 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 06 14:45:51.429834 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 06 14:45:51.429834 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:51.429834 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:51.429834 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:51.429834 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:51.429834 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:51.429834 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:51.430675 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:51.430675 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:51.430675 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:51.430675 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:51.430675 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:51.430675 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:51.430675 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:51.430675 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:51.430675 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:51.430675 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:51.430675 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 06 14:45:51.430675 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:51.430675 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 06 14:45:51.430675 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:51.430675 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 503/998] 10.222.0.23 () {56 vars in 1126 bytes} [Fri May 6 14:45:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:45:51.435948 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c62e5e9-fdd3-4e85-a007-bc8b5b03f0a8 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:51.513458 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d8139-c81c-4be6-8fad-631618cc07fd None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:51.536853 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c62e5e9-fdd3-4e85-a007-bc8b5b03f0a8 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:51.549096 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6e5586-a10e-4573-8480-6978ea3b8c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:51.549762 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e5586-a10e-4573-8480-6978ea3b8c9c None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:51.550047 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e5586-a10e-4573-8480-6978ea3b8c9c None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:51.550325 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e5586-a10e-4573-8480-6978ea3b8c9c None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:51.551219 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6e5586-a10e-4573-8480-6978ea3b8c9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:51.552588 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6e5586-a10e-4573-8480-6978ea3b8c9c None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:51.559454 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 496/999] 10.222.0.23 () {56 vars in 1087 bytes} [Fri May 6 14:45:51 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:45:51.576572 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de229d6-1330-48ca-a0b8-ccc1d1a05729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:51.577294 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de229d6-1330-48ca-a0b8-ccc1d1a05729 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:51.577579 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de229d6-1330-48ca-a0b8-ccc1d1a05729 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:51.577855 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de229d6-1330-48ca-a0b8-ccc1d1a05729 None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:51.578581 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de229d6-1330-48ca-a0b8-ccc1d1a05729 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:51.578891 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de229d6-1330-48ca-a0b8-ccc1d1a05729 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:51.580306 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de229d6-1330-48ca-a0b8-ccc1d1a05729 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:51.591249 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 504/1000] 10.222.0.23 () {56 vars in 1122 bytes} [Fri May 6 14:45:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:51.592566 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6e5586-a10e-4573-8480-6978ea3b8c9c None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:51.698324 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6e5586-a10e-4573-8480-6978ea3b8c9c None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:51.734071 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98544f0f-868f-437a-88e6-43cacdc795a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:51.734746 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98544f0f-868f-437a-88e6-43cacdc795a7 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:51.735033 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98544f0f-868f-437a-88e6-43cacdc795a7 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:51.735313 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98544f0f-868f-437a-88e6-43cacdc795a7 None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:51.736025 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98544f0f-868f-437a-88e6-43cacdc795a7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:51.736333 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98544f0f-868f-437a-88e6-43cacdc795a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:51.737739 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98544f0f-868f-437a-88e6-43cacdc795a7 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:51.743801 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 497/1001] 10.222.0.23 () {56 vars in 1122 bytes} [Fri May 6 14:45:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:51.749731 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de229d6-1330-48ca-a0b8-ccc1d1a05729 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:51.859902 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de229d6-1330-48ca-a0b8-ccc1d1a05729 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:51.893849 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771986a9-bc23-4ad4-a29a-1df8e27817f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:51.894878 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771986a9-bc23-4ad4-a29a-1df8e27817f9 None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:51.895343 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771986a9-bc23-4ad4-a29a-1df8e27817f9 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:51.895797 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771986a9-bc23-4ad4-a29a-1df8e27817f9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:51.896751 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771986a9-bc23-4ad4-a29a-1df8e27817f9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:51.898283 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771986a9-bc23-4ad4-a29a-1df8e27817f9 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:51.920656 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 505/1002] 10.222.0.23 () {60 vars in 1144 bytes} [Fri May 6 14:45:51 2022] POST /identity/v3/endpoints => generated 325 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:45:51.970428 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93ea4d-0ed1-4e5c-953f-4da48eef388e None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:51.970647 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93ea4d-0ed1-4e5c-953f-4da48eef388e None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:51.970901 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93ea4d-0ed1-4e5c-953f-4da48eef388e None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:51.972384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 498/1003] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:45:51.979148 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884dc846-ba0a-4c57-b45d-2d9db9b710ff None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:51.979465 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884dc846-ba0a-4c57-b45d-2d9db9b710ff None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:51.979710 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884dc846-ba0a-4c57-b45d-2d9db9b710ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:51.998978 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-884dc846-ba0a-4c57-b45d-2d9db9b710ff None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:52.000187 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-884dc846-ba0a-4c57-b45d-2d9db9b710ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:52.078850 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 506/1004] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:51 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:45:52.101638 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93ea4d-0ed1-4e5c-953f-4da48eef388e None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:52.207293 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93ea4d-0ed1-4e5c-953f-4da48eef388e None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:52.240838 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b39b83-91d6-4304-8dcd-c6fe90687a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:52.241635 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b39b83-91d6-4304-8dcd-c6fe90687a89 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:52.241919 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b39b83-91d6-4304-8dcd-c6fe90687a89 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:52.242197 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b39b83-91d6-4304-8dcd-c6fe90687a89 None admin] PATH_INFO: `/v3/services/image` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:52.242846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b39b83-91d6-4304-8dcd-c6fe90687a89 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:52.244222 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b39b83-91d6-4304-8dcd-c6fe90687a89 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:52.255934 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-f2b39b83-91d6-4304-8dcd-c6fe90687a89 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 06 14:45:52.255934 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:52.255934 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 06 14:45:52.255934 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 06 14:45:52.255934 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 06 14:45:52.255934 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 06 14:45:52.255934 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 06 14:45:52.255934 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:52.255934 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 06 14:45:52.255934 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:52.255934 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 06 14:45:52.255934 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:52.255934 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:52.255934 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:52.255934 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:52.255934 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:52.255934 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:52.257012 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:52.257012 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:52.257012 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:52.257012 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:52.257012 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:52.257012 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:52.257012 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 06 14:45:52.257012 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(service_id) May 06 14:45:52.257012 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 06 14:45:52.257012 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 06 14:45:52.257012 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:52.257012 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:52.257012 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:52.257012 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:52.257012 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:52.257012 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:52.257886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:52.257886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:52.257886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:52.257886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:52.257886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:52.257886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:52.257886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:52.257886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:52.257886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:52.257886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:52.257886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 06 14:45:52.257886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:52.257886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 06 14:45:52.257886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:52.257886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 499/1005] 10.222.0.23 () {56 vars in 1105 bytes} [Fri May 6 14:45:52 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:52.263394 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884dc846-ba0a-4c57-b45d-2d9db9b710ff None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:52.361517 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884dc846-ba0a-4c57-b45d-2d9db9b710ff None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:52.394283 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4573fe9-e89e-4803-92cc-71e52bbafebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:52.394951 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4573fe9-e89e-4803-92cc-71e52bbafebf None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:52.395265 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4573fe9-e89e-4803-92cc-71e52bbafebf None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:52.395539 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4573fe9-e89e-4803-92cc-71e52bbafebf None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:52.396244 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4573fe9-e89e-4803-92cc-71e52bbafebf None admin] RBAC: Adding query filter params (name=image) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:52.396556 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4573fe9-e89e-4803-92cc-71e52bbafebf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:52.397938 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4573fe9-e89e-4803-92cc-71e52bbafebf None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:52.407582 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 507/1006] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:52 2022] GET /identity/v3/services?name=image => generated 122 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:52.409020 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b39b83-91d6-4304-8dcd-c6fe90687a89 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:52.507442 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b39b83-91d6-4304-8dcd-c6fe90687a89 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:52.542228 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8074fad-dffb-40f3-8d09-55ca11983aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:52.542889 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8074fad-dffb-40f3-8d09-55ca11983aaa None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:52.543171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8074fad-dffb-40f3-8d09-55ca11983aaa None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:52.543445 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8074fad-dffb-40f3-8d09-55ca11983aaa None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:52.544380 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8074fad-dffb-40f3-8d09-55ca11983aaa None admin] RBAC: Adding query filter params (type=image) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:52.544906 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8074fad-dffb-40f3-8d09-55ca11983aaa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:52.547464 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8074fad-dffb-40f3-8d09-55ca11983aaa None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:52.556838 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 500/1007] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:52 2022] GET /identity/v3/services?type=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:53.882862 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8871eb2-e05a-4cfb-bae9-a41d1527e53e None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:53.883386 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8871eb2-e05a-4cfb-bae9-a41d1527e53e None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:53.883386 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8871eb2-e05a-4cfb-bae9-a41d1527e53e None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:53.884936 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 508/1008] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:45:53.890076 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d7c54c-c0c6-48ca-8a0d-610ab5758cf3 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:53.890454 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d7c54c-c0c6-48ca-8a0d-610ab5758cf3 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:53.890619 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d7c54c-c0c6-48ca-8a0d-610ab5758cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:53.908428 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-83d7c54c-c0c6-48ca-8a0d-610ab5758cf3 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:53.909751 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-83d7c54c-c0c6-48ca-8a0d-610ab5758cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:53.978292 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 501/1009] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:53 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:45:54.001525 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8871eb2-e05a-4cfb-bae9-a41d1527e53e None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:54.107039 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8871eb2-e05a-4cfb-bae9-a41d1527e53e None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:54.140575 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d18b8-4c2d-4c62-a5dc-4e38ba7cfaff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:54.141317 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d18b8-4c2d-4c62-a5dc-4e38ba7cfaff None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:54.141602 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d18b8-4c2d-4c62-a5dc-4e38ba7cfaff None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:54.141875 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d18b8-4c2d-4c62-a5dc-4e38ba7cfaff None admin] PATH_INFO: `/v3/services/compute` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:54.142516 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5d18b8-4c2d-4c62-a5dc-4e38ba7cfaff None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:54.143875 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5d18b8-4c2d-4c62-a5dc-4e38ba7cfaff None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:54.156296 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-cc5d18b8-4c2d-4c62-a5dc-4e38ba7cfaff None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 06 14:45:54.156296 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:54.156296 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 06 14:45:54.156296 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 06 14:45:54.156296 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 06 14:45:54.156296 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 06 14:45:54.156296 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 06 14:45:54.156296 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:54.156296 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 06 14:45:54.156296 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:54.156296 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 06 14:45:54.156296 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:54.156296 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:54.156296 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:54.156296 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:54.156296 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:54.156296 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:54.157506 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:54.157506 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:54.157506 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:54.157506 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:54.157506 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:54.157506 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:54.157506 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 06 14:45:54.157506 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(service_id) May 06 14:45:54.157506 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 06 14:45:54.157506 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 06 14:45:54.157506 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:54.157506 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:54.157506 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:54.157506 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:54.157506 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:54.157506 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:54.163906 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:54.163906 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:54.163906 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:54.163906 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:54.163906 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:54.163906 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:54.163906 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:54.163906 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:54.163906 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:54.163906 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:54.163906 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 06 14:45:54.163906 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:54.163906 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 06 14:45:54.163906 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:54.163906 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 509/1010] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:45:54 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:54.167840 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d7c54c-c0c6-48ca-8a0d-610ab5758cf3 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:54.231556 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5a7fe-ea59-47f1-8d03-ba191508c41a None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:54.231879 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5a7fe-ea59-47f1-8d03-ba191508c41a None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:54.232111 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5a7fe-ea59-47f1-8d03-ba191508c41a None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:54.233637 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 510/1011] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:45:54.238456 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c32b8-4e63-48b7-a107-d5b58c06b58c None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:54.238840 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c32b8-4e63-48b7-a107-d5b58c06b58c None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:54.239135 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c32b8-4e63-48b7-a107-d5b58c06b58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:54.255685 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-b97c32b8-4e63-48b7-a107-d5b58c06b58c None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:54.256900 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-b97c32b8-4e63-48b7-a107-d5b58c06b58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:54.265282 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d7c54c-c0c6-48ca-8a0d-610ab5758cf3 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:54.299718 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0be810-4e41-496d-b3c1-fc0185e2425a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:54.300383 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0be810-4e41-496d-b3c1-fc0185e2425a None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:54.300692 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0be810-4e41-496d-b3c1-fc0185e2425a None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:54.300971 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0be810-4e41-496d-b3c1-fc0185e2425a None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:54.301680 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0be810-4e41-496d-b3c1-fc0185e2425a None admin] RBAC: Adding query filter params (name=compute) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:54.301984 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0be810-4e41-496d-b3c1-fc0185e2425a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:54.303345 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0be810-4e41-496d-b3c1-fc0185e2425a None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:54.308177 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 502/1012] 10.222.0.23 () {56 vars in 1112 bytes} [Fri May 6 14:45:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:54.313942 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0be810-4e41-496d-b3c1-fc0185e2425a None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:54.333650 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 511/1013] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:54 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 06 14:45:54.357561 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c32b8-4e63-48b7-a107-d5b58c06b58c None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:54.413636 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0be810-4e41-496d-b3c1-fc0185e2425a None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:54.448500 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee83c9-03dc-45d9-b80c-9f87de9e85f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:54.449202 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee83c9-03dc-45d9-b80c-9f87de9e85f3 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:54.449486 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee83c9-03dc-45d9-b80c-9f87de9e85f3 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:54.449772 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee83c9-03dc-45d9-b80c-9f87de9e85f3 None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:54.450474 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ee83c9-03dc-45d9-b80c-9f87de9e85f3 None admin] RBAC: Adding query filter params (type=compute) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:54.450783 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ee83c9-03dc-45d9-b80c-9f87de9e85f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:54.452264 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ee83c9-03dc-45d9-b80c-9f87de9e85f3 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:54.455474 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c32b8-4e63-48b7-a107-d5b58c06b58c None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:54.457902 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 503/1014] 10.222.0.23 () {56 vars in 1112 bytes} [Fri May 6 14:45:54 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:54.463514 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee83c9-03dc-45d9-b80c-9f87de9e85f3 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:54.489826 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07125a0-2942-462c-9f73-cd5b9c9616ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:54.490624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07125a0-2942-462c-9f73-cd5b9c9616ff None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:54.490906 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07125a0-2942-462c-9f73-cd5b9c9616ff None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:54.491196 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07125a0-2942-462c-9f73-cd5b9c9616ff None admin] PATH_INFO: `/v3/domains/default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:54.492335 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07125a0-2942-462c-9f73-cd5b9c9616ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:54.493788 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07125a0-2942-462c-9f73-cd5b9c9616ff None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:54.495574 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 512/1015] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:54.501078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07125a0-2942-462c-9f73-cd5b9c9616ff None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:54.563130 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee83c9-03dc-45d9-b80c-9f87de9e85f3 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:54.598723 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fad6db-f2bc-4562-a664-2ffd28386ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:54.599382 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fad6db-f2bc-4562-a664-2ffd28386ede None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:54.599664 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fad6db-f2bc-4562-a664-2ffd28386ede None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:54.599984 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fad6db-f2bc-4562-a664-2ffd28386ede None admin] PATH_INFO: `/v3/endpoints` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:54.600686 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fad6db-f2bc-4562-a664-2ffd28386ede None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:54.601472 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07125a0-2942-462c-9f73-cd5b9c9616ff None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:54.602047 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fad6db-f2bc-4562-a664-2ffd28386ede None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:54.629118 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 504/1016] 10.222.0.23 () {60 vars in 1144 bytes} [Fri May 6 14:45:54 2022] POST /identity/v3/endpoints => generated 334 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 14:45:54.647070 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c746c0ba-76f7-4905-8bc3-261bf0b84b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:54.647780 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c746c0ba-76f7-4905-8bc3-261bf0b84b8d None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:54.648063 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c746c0ba-76f7-4905-8bc3-261bf0b84b8d None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:54.648339 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c746c0ba-76f7-4905-8bc3-261bf0b84b8d None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:54.649115 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c746c0ba-76f7-4905-8bc3-261bf0b84b8d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:54.650649 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c746c0ba-76f7-4905-8bc3-261bf0b84b8d None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:54.679116 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 513/1017] 10.222.0.23 () {60 vars in 1141 bytes} [Fri May 6 14:45:54 2022] POST /identity/v3/projects => generated 311 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 14:45:54.827393 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920eb81-1e79-430f-9265-4978982d6fbc None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:54.827682 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920eb81-1e79-430f-9265-4978982d6fbc None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:54.827962 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920eb81-1e79-430f-9265-4978982d6fbc None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:54.829439 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 505/1018] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:45:54.835822 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32920f-e38f-4fef-9eb8-02da112f369b None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:54.836114 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32920f-e38f-4fef-9eb8-02da112f369b None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:54.836417 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32920f-e38f-4fef-9eb8-02da112f369b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:54.854432 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-4e32920f-e38f-4fef-9eb8-02da112f369b None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:54.855660 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-4e32920f-e38f-4fef-9eb8-02da112f369b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:54.927340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 514/1019] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:54 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:45:54.951782 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2920eb81-1e79-430f-9265-4978982d6fbc None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:55.052745 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2920eb81-1e79-430f-9265-4978982d6fbc None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:55.087417 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdab0c5d-c135-4acd-8a7b-8fb628d5db90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:55.088081 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab0c5d-c135-4acd-8a7b-8fb628d5db90 None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:55.088368 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab0c5d-c135-4acd-8a7b-8fb628d5db90 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:55.088670 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab0c5d-c135-4acd-8a7b-8fb628d5db90 None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:55.089459 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdab0c5d-c135-4acd-8a7b-8fb628d5db90 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:55.090869 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdab0c5d-c135-4acd-8a7b-8fb628d5db90 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:55.106243 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 506/1020] 10.222.0.23 () {60 vars in 1141 bytes} [Fri May 6 14:45:54 2022] POST /identity/v3/services => generated 242 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 14:45:56.991920 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76744757-c193-4d83-b238-e46c854976fe None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:56.992445 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76744757-c193-4d83-b238-e46c854976fe None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:56.992445 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76744757-c193-4d83-b238-e46c854976fe None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:56.993859 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 515/1021] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:45:56.999593 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8b24c-a044-4794-b801-07e107b2d155 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:57.000006 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8b24c-a044-4794-b801-07e107b2d155 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:57.000175 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8b24c-a044-4794-b801-07e107b2d155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:57.017325 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-b9b8b24c-a044-4794-b801-07e107b2d155 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:57.018487 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b9b8b24c-a044-4794-b801-07e107b2d155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:57.039110 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdcd72b-4b8f-45b5-bae8-ebec5340a4ac None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:57.039403 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdcd72b-4b8f-45b5-bae8-ebec5340a4ac None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:57.039682 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdcd72b-4b8f-45b5-bae8-ebec5340a4ac None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:57.041111 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 516/1022] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:45:57.045938 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c3eef-d3ee-41a9-83cc-0e8961c4cede None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:57.046231 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c3eef-d3ee-41a9-83cc-0e8961c4cede None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:57.046514 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c3eef-d3ee-41a9-83cc-0e8961c4cede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:57.063998 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-791c3eef-d3ee-41a9-83cc-0e8961c4cede None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:57.065193 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-791c3eef-d3ee-41a9-83cc-0e8961c4cede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:57.092252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 507/1023] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:56 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:45:57.117333 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b8b24c-a044-4794-b801-07e107b2d155 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:57.131325 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 517/1024] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:57 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 06 14:45:57.156179 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c3eef-d3ee-41a9-83cc-0e8961c4cede None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:57.220970 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b8b24c-a044-4794-b801-07e107b2d155 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:57.255425 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2264bdb-248a-4b42-bed4-b3ef3d83082c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:57.256128 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c3eef-d3ee-41a9-83cc-0e8961c4cede None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:57.256683 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2264bdb-248a-4b42-bed4-b3ef3d83082c None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:57.256976 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2264bdb-248a-4b42-bed4-b3ef3d83082c None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:57.257250 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2264bdb-248a-4b42-bed4-b3ef3d83082c None admin] PATH_INFO: `/v3/domains/default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:57.258448 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2264bdb-248a-4b42-bed4-b3ef3d83082c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:57.259902 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2264bdb-248a-4b42-bed4-b3ef3d83082c None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:57.262058 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 508/1025] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:57.267259 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2264bdb-248a-4b42-bed4-b3ef3d83082c None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:57.291734 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d902b7c9-f8ea-42b8-af9b-ff6b9fce462f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:57.292551 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902b7c9-f8ea-42b8-af9b-ff6b9fce462f None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:57.292877 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902b7c9-f8ea-42b8-af9b-ff6b9fce462f None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:57.293158 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902b7c9-f8ea-42b8-af9b-ff6b9fce462f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:57.303343 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d902b7c9-f8ea-42b8-af9b-ff6b9fce462f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:45:57.311643 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-d902b7c9-f8ea-42b8-af9b-ff6b9fce462f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 06 14:45:57.311643 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:57.311643 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:57.311643 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:57.311643 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:57.311643 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:57.311643 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:57.311643 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:57.311643 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:57.311643 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:57.311643 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:57.311643 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:57.311643 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:45:57.311643 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:45:57.311643 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:45:57.311643 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:45:57.312706 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:45:57.312706 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:45:57.312706 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:45:57.312706 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:45:57.312706 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:57.312706 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:57.312706 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:57.312706 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:57.312706 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:57.312706 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:57.312706 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:57.312706 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:57.312706 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:57.312706 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:57.312706 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:57.312706 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:57.313611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:57.313611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:57.313611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:57.313611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:57.313611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:45:57.313611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:45:57.313611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:45:57.313611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:45:57.313611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:45:57.313611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:45:57.313611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 06 14:45:57.313611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:57.313611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 518/1026] 10.222.0.23 () {56 vars in 1120 bytes} [Fri May 6 14:45:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:57.319166 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d902b7c9-f8ea-42b8-af9b-ff6b9fce462f None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:57.369448 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2264bdb-248a-4b42-bed4-b3ef3d83082c None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:57.402445 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22911839-9daa-4852-9263-9be759055533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:57.403109 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22911839-9daa-4852-9263-9be759055533 None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:57.403393 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22911839-9daa-4852-9263-9be759055533 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:57.403669 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22911839-9daa-4852-9263-9be759055533 None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:57.404360 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22911839-9daa-4852-9263-9be759055533 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:57.406082 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22911839-9daa-4852-9263-9be759055533 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:57.415269 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d902b7c9-f8ea-42b8-af9b-ff6b9fce462f None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:57.424733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 509/1027] 10.222.0.23 () {60 vars in 1132 bytes} [Fri May 6 14:45:57 2022] POST /identity/v3/users => generated 285 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:45:57.426089 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6685d35b-b460-47fc-811d-f5a37a8326d9 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:57.426364 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6685d35b-b460-47fc-811d-f5a37a8326d9 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:57.426643 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6685d35b-b460-47fc-811d-f5a37a8326d9 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:57.428054 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 510/1028] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:45:57.433782 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697e4bb-df91-4cd5-9e8b-d31614f9b93b None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:57.434108 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697e4bb-df91-4cd5-9e8b-d31614f9b93b None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:57.434349 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697e4bb-df91-4cd5-9e8b-d31614f9b93b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:57.451387 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6199f8-7089-4f1c-990a-f29ff77b7351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:57.452077 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6199f8-7089-4f1c-990a-f29ff77b7351 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:57.452360 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6199f8-7089-4f1c-990a-f29ff77b7351 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:57.452659 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6199f8-7089-4f1c-990a-f29ff77b7351 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:57.452763 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-e697e4bb-df91-4cd5-9e8b-d31614f9b93b None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:57.453415 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6199f8-7089-4f1c-990a-f29ff77b7351 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:57.453720 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6199f8-7089-4f1c-990a-f29ff77b7351 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:57.453971 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-e697e4bb-df91-4cd5-9e8b-d31614f9b93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:57.455190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6199f8-7089-4f1c-990a-f29ff77b7351 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:57.465028 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 519/1029] 10.222.0.23 () {56 vars in 1115 bytes} [Fri May 6 14:45:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:57.470712 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6199f8-7089-4f1c-990a-f29ff77b7351 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:57.530492 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 511/1030] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:57 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 06 14:45:57.557942 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e697e4bb-df91-4cd5-9e8b-d31614f9b93b None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:57.584314 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6199f8-7089-4f1c-990a-f29ff77b7351 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:57.620248 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc52d5-19a4-4f30-85c9-7e6b7933af9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:57.621072 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc52d5-19a4-4f30-85c9-7e6b7933af9e None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:57.621355 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc52d5-19a4-4f30-85c9-7e6b7933af9e None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:57.621627 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc52d5-19a4-4f30-85c9-7e6b7933af9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:57.633667 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cc52d5-19a4-4f30-85c9-7e6b7933af9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:57.635121 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cc52d5-19a4-4f30-85c9-7e6b7933af9e None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:57.652474 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-e3cc52d5-19a4-4f30-85c9-7e6b7933af9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:57.652474 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:57.652474 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:57.652474 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:57.652474 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:57.652474 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:57.652474 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:57.652474 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:57.652474 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:57.652474 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:57.652474 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:57.652474 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:57.652474 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:57.652474 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:57.652474 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:57.653552 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:57.653552 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:57.653552 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:57.653552 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:57.653552 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:57.653552 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:57.653552 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:57.653552 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:57.653552 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:57.653552 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:57.653552 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:57.653552 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:57.653552 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:57.653552 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:57.653552 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:57.653552 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:57.653552 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:57.654896 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:57.654896 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:57.654896 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:57.654896 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:57.654896 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 520/1031] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:45:57.659507 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc52d5-19a4-4f30-85c9-7e6b7933af9e None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:57.664889 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e697e4bb-df91-4cd5-9e8b-d31614f9b93b None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:57.704240 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5bb904-9944-43ad-aae7-6806492b30cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:57.705059 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5bb904-9944-43ad-aae7-6806492b30cc None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:57.705339 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5bb904-9944-43ad-aae7-6806492b30cc None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:57.705610 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5bb904-9944-43ad-aae7-6806492b30cc None admin] PATH_INFO: `/v3/services/image` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:57.706371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5bb904-9944-43ad-aae7-6806492b30cc None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:57.708449 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5bb904-9944-43ad-aae7-6806492b30cc None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:57.720326 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-3c5bb904-9944-43ad-aae7-6806492b30cc None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 06 14:45:57.720326 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:57.720326 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 06 14:45:57.720326 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 06 14:45:57.720326 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 06 14:45:57.720326 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 06 14:45:57.720326 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 06 14:45:57.720326 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:57.720326 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 06 14:45:57.720326 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:57.720326 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 06 14:45:57.720326 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:57.720326 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:57.720326 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:57.720326 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:57.720326 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:57.720326 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:57.721541 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:57.721541 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:57.721541 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:57.721541 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:57.721541 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:57.721541 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:57.721541 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 06 14:45:57.721541 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_service(service_id) May 06 14:45:57.721541 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 06 14:45:57.721541 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 06 14:45:57.721541 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:57.721541 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:57.721541 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:57.721541 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:57.721541 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:57.721541 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:57.723147 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:57.723147 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:57.723147 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:57.723147 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:57.723147 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:57.723147 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:57.723147 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:57.723147 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:57.723147 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:57.723147 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:57.723147 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 06 14:45:57.723147 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:45:57.723147 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 06 14:45:57.723147 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:57.723147 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 512/1032] 10.222.0.23 () {56 vars in 1105 bytes} [Fri May 6 14:45:57 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:45:57.727365 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5bb904-9944-43ad-aae7-6806492b30cc None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:57.759471 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc52d5-19a4-4f30-85c9-7e6b7933af9e None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:57.793812 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56522ff6-f76e-4ca4-a400-87c02abee144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:57.794423 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56522ff6-f76e-4ca4-a400-87c02abee144 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:57.794713 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56522ff6-f76e-4ca4-a400-87c02abee144 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:57.794992 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56522ff6-f76e-4ca4-a400-87c02abee144 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:57.795760 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56522ff6-f76e-4ca4-a400-87c02abee144 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:57.796073 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56522ff6-f76e-4ca4-a400-87c02abee144 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:57.797471 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56522ff6-f76e-4ca4-a400-87c02abee144 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:57.810909 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 521/1033] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:57.815428 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56522ff6-f76e-4ca4-a400-87c02abee144 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:57.830892 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5bb904-9944-43ad-aae7-6806492b30cc None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:57.863566 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73252e30-cd32-4bbe-98f6-b7f70fb81305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:57.864580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73252e30-cd32-4bbe-98f6-b7f70fb81305 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:57.864889 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73252e30-cd32-4bbe-98f6-b7f70fb81305 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:57.865212 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73252e30-cd32-4bbe-98f6-b7f70fb81305 None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:57.865919 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73252e30-cd32-4bbe-98f6-b7f70fb81305 None admin] RBAC: Adding query filter params (name=image) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:57.866222 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73252e30-cd32-4bbe-98f6-b7f70fb81305 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:57.867585 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73252e30-cd32-4bbe-98f6-b7f70fb81305 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:57.872566 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 513/1034] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:57 2022] GET /identity/v3/services?name=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:57.877837 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73252e30-cd32-4bbe-98f6-b7f70fb81305 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:57.915947 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56522ff6-f76e-4ca4-a400-87c02abee144 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:57.955621 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32df8e-0dce-4eeb-b59e-97ddf8c786ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:57.956411 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32df8e-0dce-4eeb-b59e-97ddf8c786ee None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:57.956889 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32df8e-0dce-4eeb-b59e-97ddf8c786ee None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:57.957223 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32df8e-0dce-4eeb-b59e-97ddf8c786ee None admin] PATH_INFO: `/v3/users/nova` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:57.964629 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e32df8e-0dce-4eeb-b59e-97ddf8c786ee None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:57.966049 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e32df8e-0dce-4eeb-b59e-97ddf8c786ee None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:57.976398 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73252e30-cd32-4bbe-98f6-b7f70fb81305 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:57.980894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-2e32df8e-0dce-4eeb-b59e-97ddf8c786ee None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 06 14:45:57.980894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:57.980894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:57.980894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:57.980894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:57.980894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:57.980894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:57.980894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:57.980894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:57.980894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:57.980894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:57.980894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:57.980894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 06 14:45:57.980894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_user(user_id) May 06 14:45:57.980894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 06 14:45:57.980894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 06 14:45:57.981823 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:57.981823 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:57.981823 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 06 14:45:57.981823 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:57.981823 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 06 14:45:57.981823 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:45:57.981823 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:57.981823 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:57.981823 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:57.981823 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:57.981823 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:57.981823 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:45:57.981823 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:57.981823 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:57.981823 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:57.981823 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:57.981823 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:57.981823 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:57.982771 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:57.982771 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:57.982771 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 06 14:45:57.982771 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 06 14:45:57.982771 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 06 14:45:57.982771 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 06 14:45:57.982771 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 06 14:45:57.982771 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 06 14:45:57.982771 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 06 14:45:57.982771 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:45:57.984972 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 522/1035] 10.222.0.23 () {56 vars in 1093 bytes} [Fri May 6 14:45:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:45:57.987546 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32df8e-0dce-4eeb-b59e-97ddf8c786ee None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:58.012187 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b35d563-ceb5-45fe-9b94-eb1f700d81c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:58.012883 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b35d563-ceb5-45fe-9b94-eb1f700d81c9 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:58.013152 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b35d563-ceb5-45fe-9b94-eb1f700d81c9 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:58.013428 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b35d563-ceb5-45fe-9b94-eb1f700d81c9 None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:58.014123 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b35d563-ceb5-45fe-9b94-eb1f700d81c9 None admin] RBAC: Adding query filter params (type=image) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:58.014428 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b35d563-ceb5-45fe-9b94-eb1f700d81c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:58.015781 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b35d563-ceb5-45fe-9b94-eb1f700d81c9 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:58.020728 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 514/1036] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:57 2022] GET /identity/v3/services?type=image => generated 350 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:58.026671 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b35d563-ceb5-45fe-9b94-eb1f700d81c9 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:58.088566 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32df8e-0dce-4eeb-b59e-97ddf8c786ee None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:58.122224 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1feb669-306e-4322-a49c-cac4dd125127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:58.122903 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1feb669-306e-4322-a49c-cac4dd125127 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:58.123186 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1feb669-306e-4322-a49c-cac4dd125127 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:58.123463 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1feb669-306e-4322-a49c-cac4dd125127 None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:58.124283 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1feb669-306e-4322-a49c-cac4dd125127 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:58.124508 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b35d563-ceb5-45fe-9b94-eb1f700d81c9 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:58.124725 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1feb669-306e-4322-a49c-cac4dd125127 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:58.126030 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1feb669-306e-4322-a49c-cac4dd125127 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:58.143150 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 523/1037] 10.222.0.23 () {56 vars in 1133 bytes} [Fri May 6 14:45:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:58.148826 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1feb669-306e-4322-a49c-cac4dd125127 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:58.160352 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6a6792-8eaf-4730-9df4-e30c653ea536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:58.161047 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a6792-8eaf-4730-9df4-e30c653ea536 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:58.161324 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a6792-8eaf-4730-9df4-e30c653ea536 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:58.161598 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a6792-8eaf-4730-9df4-e30c653ea536 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:58.162821 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6a6792-8eaf-4730-9df4-e30c653ea536 None admin] RBAC: Adding query filter params (interface=public, service_id=baf5b1711b7343c58000dc6acfde6a37, region_id=RegionOne) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:58.163151 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6a6792-8eaf-4730-9df4-e30c653ea536 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:58.164868 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6a6792-8eaf-4730-9df4-e30c653ea536 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:58.170106 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 515/1038] 10.222.0.23 () {56 vars in 1251 bytes} [Fri May 6 14:45:58 2022] GET /identity/v3/endpoints?service_id=baf5b1711b7343c58000dc6acfde6a37&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:58.175998 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6a6792-8eaf-4730-9df4-e30c653ea536 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:58.258146 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1feb669-306e-4322-a49c-cac4dd125127 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:58.286247 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6a6792-8eaf-4730-9df4-e30c653ea536 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:58.295396 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca9c01-d0bc-4831-bf3a-ef2e41c62b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:58.296563 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca9c01-d0bc-4831-bf3a-ef2e41c62b71 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:58.297264 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca9c01-d0bc-4831-bf3a-ef2e41c62b71 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:58.297687 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca9c01-d0bc-4831-bf3a-ef2e41c62b71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:58.320366 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe32f0-8033-4143-8eb7-f3ca9b091303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:58.321143 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe32f0-8033-4143-8eb7-f3ca9b091303 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:58.321423 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe32f0-8033-4143-8eb7-f3ca9b091303 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:58.321699 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe32f0-8033-4143-8eb7-f3ca9b091303 None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:58.322378 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbe32f0-8033-4143-8eb7-f3ca9b091303 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:58.323731 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbe32f0-8033-4143-8eb7-f3ca9b091303 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:58.330195 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 516/1039] 10.222.0.23 () {56 vars in 1087 bytes} [Fri May 6 14:45:58 2022] GET /identity/v3/services => generated 1957 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 06 14:45:58.376730 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ca9c01-d0bc-4831-bf3a-ef2e41c62b71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:58.379006 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ca9c01-d0bc-4831-bf3a-ef2e41c62b71 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:58.490276 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 524/1040] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:45:58 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 06 14:45:58.492148 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe32f0-8033-4143-8eb7-f3ca9b091303 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:58.596236 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe32f0-8033-4143-8eb7-f3ca9b091303 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:58.630552 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce37c7-82fa-4109-b1ad-c7a0853d796a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:58.631368 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce37c7-82fa-4109-b1ad-c7a0853d796a None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:58.631664 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce37c7-82fa-4109-b1ad-c7a0853d796a None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:58.631956 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce37c7-82fa-4109-b1ad-c7a0853d796a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:58.642963 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ce37c7-82fa-4109-b1ad-c7a0853d796a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:58.644419 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ce37c7-82fa-4109-b1ad-c7a0853d796a None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:58.661095 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-c1ce37c7-82fa-4109-b1ad-c7a0853d796a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:58.661095 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:58.661095 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:58.661095 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:58.661095 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:58.661095 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:58.661095 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:58.661095 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:58.661095 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:58.661095 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:58.661095 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:58.661095 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:58.661095 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:45:58.661095 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:45:58.661095 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:45:58.662077 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:45:58.662077 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:58.662077 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:58.662077 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:58.662077 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:58.662077 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:58.662077 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:58.662077 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:58.662077 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:58.662077 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:58.662077 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:58.662077 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:58.662077 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:58.662077 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:58.662077 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:58.662077 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:58.662077 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:58.662994 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:45:58.662994 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:45:58.662994 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:45:58.662994 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:58.662994 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 517/1041] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:45:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:45:58.667874 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca9c01-d0bc-4831-bf3a-ef2e41c62b71 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:58.763719 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca9c01-d0bc-4831-bf3a-ef2e41c62b71 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:58.796208 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f6da4-e04a-4819-9b25-eb823ce81dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:58.796899 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f6da4-e04a-4819-9b25-eb823ce81dcf None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:58.797183 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f6da4-e04a-4819-9b25-eb823ce81dcf None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:58.797459 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f6da4-e04a-4819-9b25-eb823ce81dcf None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:58.798164 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606f6da4-e04a-4819-9b25-eb823ce81dcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:58.798467 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606f6da4-e04a-4819-9b25-eb823ce81dcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:58.799809 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606f6da4-e04a-4819-9b25-eb823ce81dcf None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:58.817528 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 525/1042] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:45:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:58.819461 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce37c7-82fa-4109-b1ad-c7a0853d796a None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:58.916371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce37c7-82fa-4109-b1ad-c7a0853d796a None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:58.948760 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c0587-a937-4c49-a387-aee128c05936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:58.949542 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c0587-a937-4c49-a387-aee128c05936 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:58.949823 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c0587-a937-4c49-a387-aee128c05936 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:58.950096 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c0587-a937-4c49-a387-aee128c05936 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:58.956631 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83c0587-a937-4c49-a387-aee128c05936 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:58.958066 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83c0587-a937-4c49-a387-aee128c05936 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:58.971827 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-b83c0587-a937-4c49-a387-aee128c05936 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:58.971827 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:45:58.971827 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:45:58.971827 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:45:58.971827 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:45:58.971827 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:45:58.971827 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:45:58.971827 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:45:58.971827 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:45:58.971827 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:45:58.971827 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:45:58.971827 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:45:58.971827 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 06 14:45:58.971827 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(project_id) May 06 14:45:58.971827 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 06 14:45:58.973355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 06 14:45:58.973355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:45:58.973355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:45:58.973355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:45:58.973355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:45:58.973355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:45:58.973355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:45:58.973355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:45:58.973355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:45:58.973355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:45:58.973355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:45:58.973355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:45:58.973355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:45:58.973355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:45:58.973355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:45:58.973355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:45:58.973355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:45:58.980574 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 06 14:45:58.980574 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 06 14:45:58.980574 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 06 14:45:58.980574 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 06 14:45:58.980574 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 06 14:45:58.980574 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 06 14:45:58.980574 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:45:58.980574 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:45:58.980574 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 518/1043] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:45:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:45:58.981205 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f6da4-e04a-4819-9b25-eb823ce81dcf None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:59.075825 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f6da4-e04a-4819-9b25-eb823ce81dcf None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:59.108120 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d01a61e-e39b-415c-80f2-ade5c148c56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:59.108841 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01a61e-e39b-415c-80f2-ade5c148c56e None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:59.109086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01a61e-e39b-415c-80f2-ade5c148c56e None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:59.109346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01a61e-e39b-415c-80f2-ade5c148c56e None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:59.110114 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d01a61e-e39b-415c-80f2-ade5c148c56e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:59.110418 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d01a61e-e39b-415c-80f2-ade5c148c56e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:59.111837 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d01a61e-e39b-415c-80f2-ade5c148c56e None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:59.129702 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 526/1044] 10.222.0.23 () {56 vars in 1148 bytes} [Fri May 6 14:45:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:45:59.131137 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c0587-a937-4c49-a387-aee128c05936 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:59.234578 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c0587-a937-4c49-a387-aee128c05936 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:45:59.268078 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5653588d-613c-4ebc-9043-e1660677d52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:45:59.268769 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653588d-613c-4ebc-9043-e1660677d52a None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:59.269864 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653588d-613c-4ebc-9043-e1660677d52a None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:59.270135 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653588d-613c-4ebc-9043-e1660677d52a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:59.270957 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5653588d-613c-4ebc-9043-e1660677d52a None admin] RBAC: Adding query filter params (role.id=5213da7472024e87a41cecfaa37abce3, scope.project.id=d5398feee6eb490cb49bb261448a1a9b, user.id=fc8cf09642ec4e10a80badcf8b585ef8) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:45:59.271266 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5653588d-613c-4ebc-9043-e1660677d52a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:45:59.272713 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5653588d-613c-4ebc-9043-e1660677d52a None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:45:59.285430 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 519/1045] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:45:59 2022] GET /identity/v3/role_assignments?user.id=fc8cf09642ec4e10a80badcf8b585ef8&scope.project.id=d5398feee6eb490cb49bb261448a1a9b&role.id=5213da7472024e87a41cecfaa37abce3 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:45:59.757486 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb66587-c457-483f-8338-4b48b19433c1 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:59.757812 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb66587-c457-483f-8338-4b48b19433c1 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:59.758057 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb66587-c457-483f-8338-4b48b19433c1 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:59.759564 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 527/1046] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:45:59.764811 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff482bd6-b64f-49c5-b1dd-ffa223ae97b6 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:45:59.765107 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff482bd6-b64f-49c5-b1dd-ffa223ae97b6 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:45:59.765387 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff482bd6-b64f-49c5-b1dd-ffa223ae97b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:45:59.781939 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-ff482bd6-b64f-49c5-b1dd-ffa223ae97b6 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:45:59.783114 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ff482bd6-b64f-49c5-b1dd-ffa223ae97b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:45:59.848836 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 520/1047] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:45:59 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:45:59.871900 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb66587-c457-483f-8338-4b48b19433c1 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:45:59.970149 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb66587-c457-483f-8338-4b48b19433c1 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:00.004048 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ffa563-37d3-43c8-9060-d5fa589a08cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:00.004893 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ffa563-37d3-43c8-9060-d5fa589a08cc None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:00.005178 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ffa563-37d3-43c8-9060-d5fa589a08cc None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:00.005450 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ffa563-37d3-43c8-9060-d5fa589a08cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:00.015354 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ffa563-37d3-43c8-9060-d5fa589a08cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:46:00.023221 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-35ffa563-37d3-43c8-9060-d5fa589a08cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:46:00.023221 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:46:00.023221 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:46:00.023221 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:46:00.023221 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:46:00.023221 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:46:00.023221 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:46:00.023221 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:46:00.023221 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:46:00.023221 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:46:00.023221 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:46:00.023221 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:46:00.023221 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:46:00.023221 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:46:00.023221 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:46:00.023221 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:46:00.024204 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:46:00.024204 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:46:00.024204 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:46:00.024204 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:46:00.024204 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:46:00.024204 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:46:00.024204 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:46:00.024204 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:46:00.024204 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:46:00.024204 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:46:00.024204 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:46:00.024204 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:46:00.024204 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:46:00.024204 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:46:00.024204 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:46:00.024204 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:46:00.025115 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:46:00.025115 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:46:00.025115 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:46:00.025115 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:46:00.025115 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:46:00.025115 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:46:00.025115 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:46:00.025115 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:46:00.025115 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:46:00.025115 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:46:00.025115 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:46:00.025115 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:46:00.028926 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 528/1048] 10.222.0.23 () {56 vars in 1096 bytes} [Fri May 6 14:45:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:46:00.030808 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff482bd6-b64f-49c5-b1dd-ffa223ae97b6 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:00.125307 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff482bd6-b64f-49c5-b1dd-ffa223ae97b6 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:00.165970 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb58ba1a-1cd2-400c-b00f-0b5b5b939dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:00.166643 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb58ba1a-1cd2-400c-b00f-0b5b5b939dbe None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:00.166918 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb58ba1a-1cd2-400c-b00f-0b5b5b939dbe None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:00.167193 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb58ba1a-1cd2-400c-b00f-0b5b5b939dbe None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:00.167930 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb58ba1a-1cd2-400c-b00f-0b5b5b939dbe None admin] RBAC: Adding query filter params (name=admin) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:46:00.168234 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb58ba1a-1cd2-400c-b00f-0b5b5b939dbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:00.169614 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb58ba1a-1cd2-400c-b00f-0b5b5b939dbe None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:00.182834 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 521/1049] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:46:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:00.184354 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ffa563-37d3-43c8-9060-d5fa589a08cc None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:00.282435 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ffa563-37d3-43c8-9060-d5fa589a08cc None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:00.314518 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7257eb7b-fcf2-4ed3-946a-b29b2bd11d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:00.315390 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257eb7b-fcf2-4ed3-946a-b29b2bd11d8f None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:00.315675 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257eb7b-fcf2-4ed3-946a-b29b2bd11d8f None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:00.316196 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257eb7b-fcf2-4ed3-946a-b29b2bd11d8f None admin] PATH_INFO: `/v3/users/d087368a067748179c11ed9d6e06711d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:00.330019 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7257eb7b-fcf2-4ed3-946a-b29b2bd11d8f None admin] RBAC: Authorizing `identity:get_user(user_id=d087368a067748179c11ed9d6e06711d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:00.331997 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7257eb7b-fcf2-4ed3-946a-b29b2bd11d8f None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:00.339815 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 529/1050] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:46:00 2022] GET /identity/v3/users/d087368a067748179c11ed9d6e06711d => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:00.339943 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb58ba1a-1cd2-400c-b00f-0b5b5b939dbe None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:00.442252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb58ba1a-1cd2-400c-b00f-0b5b5b939dbe None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:00.475390 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76621ead-4d63-430b-9373-8ada844d1d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:00.476040 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76621ead-4d63-430b-9373-8ada844d1d94 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:00.476323 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76621ead-4d63-430b-9373-8ada844d1d94 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:00.476613 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76621ead-4d63-430b-9373-8ada844d1d94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:00.544093 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76621ead-4d63-430b-9373-8ada844d1d94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:00.545493 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76621ead-4d63-430b-9373-8ada844d1d94 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:00.599877 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0baa1d-79d5-4b24-949c-75861bc36d05 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:00.600197 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0baa1d-79d5-4b24-949c-75861bc36d05 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:00.600427 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0baa1d-79d5-4b24-949c-75861bc36d05 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:00.601941 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 530/1051] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:46:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:46:00.606591 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781261e-ae39-45bb-ad9c-43399f97e427 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:00.606875 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781261e-ae39-45bb-ad9c-43399f97e427 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:00.607150 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781261e-ae39-45bb-ad9c-43399f97e427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:00.623238 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-5781261e-ae39-45bb-ad9c-43399f97e427 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:46:00.624522 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-5781261e-ae39-45bb-ad9c-43399f97e427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:46:00.648051 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 522/1052] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:46:00 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 06 14:46:00.650045 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76621ead-4d63-430b-9373-8ada844d1d94 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:00.689892 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 531/1053] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:46:00 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 06 14:46:00.713482 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5781261e-ae39-45bb-ad9c-43399f97e427 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:00.754731 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76621ead-4d63-430b-9373-8ada844d1d94 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:00.788996 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b3011-b9dd-4277-9540-f72691e4340a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:00.789855 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b3011-b9dd-4277-9540-f72691e4340a None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:00.790135 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b3011-b9dd-4277-9540-f72691e4340a None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:00.790414 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b3011-b9dd-4277-9540-f72691e4340a None admin] PATH_INFO: `/v3/projects/4456b4addd3d46fa9e8410b04869c47e` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:00.791579 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6b3011-b9dd-4277-9540-f72691e4340a None admin] RBAC: Authorizing `identity:get_project(project_id=4456b4addd3d46fa9e8410b04869c47e)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:00.793043 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6b3011-b9dd-4277-9540-f72691e4340a None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:00.794775 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 523/1054] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:46:00 2022] GET /identity/v3/projects/4456b4addd3d46fa9e8410b04869c47e => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:00.800670 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b3011-b9dd-4277-9540-f72691e4340a None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:00.816440 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5781261e-ae39-45bb-ad9c-43399f97e427 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:00.850763 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eafd775-436d-42da-b22e-78e13383cab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:00.851563 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafd775-436d-42da-b22e-78e13383cab1 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:00.851848 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafd775-436d-42da-b22e-78e13383cab1 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:00.852123 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafd775-436d-42da-b22e-78e13383cab1 None admin] PATH_INFO: `/v3/services/image` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:00.852802 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eafd775-436d-42da-b22e-78e13383cab1 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:00.854160 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eafd775-436d-42da-b22e-78e13383cab1 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:00.865599 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-5eafd775-436d-42da-b22e-78e13383cab1 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 06 14:46:00.865599 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:46:00.865599 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 06 14:46:00.865599 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 06 14:46:00.865599 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 06 14:46:00.865599 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 06 14:46:00.865599 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 06 14:46:00.865599 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:46:00.865599 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 06 14:46:00.865599 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:46:00.865599 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 06 14:46:00.865599 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:46:00.865599 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:46:00.865599 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:46:00.865599 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:46:00.865599 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:46:00.865599 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:46:00.866636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:46:00.866636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:46:00.866636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:46:00.866636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:46:00.866636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:46:00.866636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:46:00.866636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 06 14:46:00.866636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(service_id) May 06 14:46:00.866636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 06 14:46:00.866636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 06 14:46:00.866636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:46:00.866636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:46:00.866636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:46:00.866636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:46:00.866636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:46:00.866636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:46:00.867483 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:46:00.867483 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:46:00.867483 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:46:00.867483 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:46:00.867483 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:46:00.867483 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:46:00.867483 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:46:00.867483 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:46:00.867483 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:46:00.867483 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:46:00.867483 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 06 14:46:00.867483 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:46:00.867483 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 06 14:46:00.867483 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:46:00.867483 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 532/1055] 10.222.0.23 () {56 vars in 1105 bytes} [Fri May 6 14:46:00 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:46:00.872830 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eafd775-436d-42da-b22e-78e13383cab1 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:00.904084 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b3011-b9dd-4277-9540-f72691e4340a None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:00.938200 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc73aa-dd97-42c0-88bd-a9115e74fa18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:00.938856 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc73aa-dd97-42c0-88bd-a9115e74fa18 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:00.939139 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc73aa-dd97-42c0-88bd-a9115e74fa18 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:00.939416 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc73aa-dd97-42c0-88bd-a9115e74fa18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:00.940246 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cc73aa-dd97-42c0-88bd-a9115e74fa18 None admin] RBAC: Adding query filter params (role.id=6dcd35d1a83748759d6ef9f026077835, scope.project.id=4456b4addd3d46fa9e8410b04869c47e, user.id=d087368a067748179c11ed9d6e06711d) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:46:00.940559 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cc73aa-dd97-42c0-88bd-a9115e74fa18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:00.942204 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cc73aa-dd97-42c0-88bd-a9115e74fa18 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:00.948722 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 524/1056] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:46:00 2022] GET /identity/v3/role_assignments?user.id=d087368a067748179c11ed9d6e06711d&scope.project.id=4456b4addd3d46fa9e8410b04869c47e&role.id=6dcd35d1a83748759d6ef9f026077835 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:46:00.970914 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eafd775-436d-42da-b22e-78e13383cab1 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:01.006672 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964ad68-7225-488a-89c3-b1db5c9a3d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:01.007366 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964ad68-7225-488a-89c3-b1db5c9a3d62 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:01.007654 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964ad68-7225-488a-89c3-b1db5c9a3d62 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:01.007930 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964ad68-7225-488a-89c3-b1db5c9a3d62 None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:01.008680 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6964ad68-7225-488a-89c3-b1db5c9a3d62 None admin] RBAC: Adding query filter params (name=image) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:46:01.008993 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6964ad68-7225-488a-89c3-b1db5c9a3d62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:01.010399 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6964ad68-7225-488a-89c3-b1db5c9a3d62 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:01.021060 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 533/1057] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:46:00 2022] GET /identity/v3/services?name=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:46:01.022855 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc73aa-dd97-42c0-88bd-a9115e74fa18 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:01.128120 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc73aa-dd97-42c0-88bd-a9115e74fa18 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:01.165601 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d435aecd-d25f-4f1a-b898-48b432c894ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:01.166266 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d435aecd-d25f-4f1a-b898-48b432c894ca None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:01.166737 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d435aecd-d25f-4f1a-b898-48b432c894ca None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:01.166813 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d435aecd-d25f-4f1a-b898-48b432c894ca None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:01.167483 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d435aecd-d25f-4f1a-b898-48b432c894ca None admin] RBAC: Adding query filter params (type=image) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:46:01.167782 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d435aecd-d25f-4f1a-b898-48b432c894ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:01.169277 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d435aecd-d25f-4f1a-b898-48b432c894ca None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:01.175149 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 525/1058] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:46:01 2022] GET /identity/v3/services?type=image => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:46:01.180734 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964ad68-7225-488a-89c3-b1db5c9a3d62 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:01.276890 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964ad68-7225-488a-89c3-b1db5c9a3d62 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:01.308993 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e446e747-367d-4e3a-b2f8-33a44b19f11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:01.309649 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446e747-367d-4e3a-b2f8-33a44b19f11c None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:01.309927 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446e747-367d-4e3a-b2f8-33a44b19f11c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:01.310195 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446e747-367d-4e3a-b2f8-33a44b19f11c None admin] PATH_INFO: `/v3/endpoints` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:01.310925 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e446e747-367d-4e3a-b2f8-33a44b19f11c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:01.312270 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e446e747-367d-4e3a-b2f8-33a44b19f11c None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:01.332715 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 534/1059] 10.222.0.23 () {60 vars in 1144 bytes} [Fri May 6 14:46:01 2022] POST /identity/v3/endpoints => generated 327 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:46:01.552304 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bafa44-d496-46ed-b8ea-8f6db4594830 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:01.552573 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bafa44-d496-46ed-b8ea-8f6db4594830 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:01.552899 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bafa44-d496-46ed-b8ea-8f6db4594830 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:01.554235 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 526/1060] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:46:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:46:01.559744 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00766065-0b73-42dd-ab4c-55a7dbaa56ae None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:01.560324 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00766065-0b73-42dd-ab4c-55a7dbaa56ae None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:01.560848 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00766065-0b73-42dd-ab4c-55a7dbaa56ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:01.578174 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-00766065-0b73-42dd-ab4c-55a7dbaa56ae None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:46:01.579340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-00766065-0b73-42dd-ab4c-55a7dbaa56ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:46:01.649873 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 535/1061] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:46:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:46:01.672267 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bafa44-d496-46ed-b8ea-8f6db4594830 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:01.771172 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bafa44-d496-46ed-b8ea-8f6db4594830 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:01.804706 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5e4249-32bd-40e8-afb2-dbe70a8baa31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:01.805427 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e4249-32bd-40e8-afb2-dbe70a8baa31 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:01.805706 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e4249-32bd-40e8-afb2-dbe70a8baa31 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:01.805976 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e4249-32bd-40e8-afb2-dbe70a8baa31 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:01.816671 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5e4249-32bd-40e8-afb2-dbe70a8baa31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:46:01.824832 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-3e5e4249-32bd-40e8-afb2-dbe70a8baa31 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 06 14:46:01.824832 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:46:01.824832 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:46:01.824832 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:46:01.824832 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:46:01.824832 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:46:01.824832 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:46:01.824832 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:46:01.824832 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:46:01.824832 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:46:01.824832 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:46:01.824832 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:46:01.824832 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:46:01.824832 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:46:01.824832 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:46:01.824832 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:46:01.825756 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:46:01.825756 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:46:01.825756 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:46:01.825756 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:46:01.825756 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:46:01.825756 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:46:01.825756 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:46:01.825756 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:46:01.825756 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:46:01.825756 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:46:01.825756 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:46:01.825756 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:46:01.825756 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:46:01.825756 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:46:01.825756 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:46:01.825756 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:46:01.826616 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:46:01.826616 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:46:01.826616 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:46:01.826616 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:46:01.826616 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:46:01.826616 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:46:01.826616 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:46:01.826616 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:46:01.826616 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:46:01.826616 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:46:01.826616 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 06 14:46:01.826616 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:46:01.829340 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 527/1062] 10.222.0.23 () {56 vars in 1120 bytes} [Fri May 6 14:46:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:46:01.832211 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00766065-0b73-42dd-ab4c-55a7dbaa56ae None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:01.927520 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00766065-0b73-42dd-ab4c-55a7dbaa56ae None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:01.960898 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a04a61-85b6-4729-bf4c-81b7c59de971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:01.961571 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a04a61-85b6-4729-bf4c-81b7c59de971 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:01.961853 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a04a61-85b6-4729-bf4c-81b7c59de971 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:01.962127 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a04a61-85b6-4729-bf4c-81b7c59de971 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:01.962860 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a04a61-85b6-4729-bf4c-81b7c59de971 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:46:01.963238 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a04a61-85b6-4729-bf4c-81b7c59de971 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:01.964611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a04a61-85b6-4729-bf4c-81b7c59de971 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:01.977310 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 536/1063] 10.222.0.23 () {56 vars in 1115 bytes} [Fri May 6 14:46:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:01.978512 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5e4249-32bd-40e8-afb2-dbe70a8baa31 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:02.079603 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5e4249-32bd-40e8-afb2-dbe70a8baa31 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:02.115378 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70750c2c-7ab5-4847-ae69-bc228598de36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:02.116176 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70750c2c-7ab5-4847-ae69-bc228598de36 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:02.116507 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70750c2c-7ab5-4847-ae69-bc228598de36 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:02.116814 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70750c2c-7ab5-4847-ae69-bc228598de36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:02.127642 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70750c2c-7ab5-4847-ae69-bc228598de36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:02.129884 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70750c2c-7ab5-4847-ae69-bc228598de36 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:02.147452 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-70750c2c-7ab5-4847-ae69-bc228598de36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:46:02.147452 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:46:02.147452 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:46:02.147452 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:46:02.147452 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:46:02.147452 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:46:02.147452 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:46:02.147452 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:46:02.147452 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:46:02.147452 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:46:02.147452 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:46:02.147452 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:46:02.147452 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:46:02.147452 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:46:02.147452 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:46:02.148363 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:46:02.148363 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:46:02.148363 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:46:02.148363 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:46:02.148363 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:46:02.148363 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:46:02.148363 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:46:02.148363 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:46:02.148363 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:46:02.148363 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:46:02.148363 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:46:02.148363 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:46:02.148363 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:46:02.148363 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:46:02.148363 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:46:02.148363 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:46:02.148363 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:46:02.149315 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:46:02.149315 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:46:02.149315 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:46:02.149315 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:46:02.149315 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 528/1064] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:46:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:46:02.155721 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a04a61-85b6-4729-bf4c-81b7c59de971 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:02.255040 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a04a61-85b6-4729-bf4c-81b7c59de971 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:02.286767 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed13b6-48e2-4a5c-bf56-e4d05da069c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:02.287480 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed13b6-48e2-4a5c-bf56-e4d05da069c8 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:02.287711 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed13b6-48e2-4a5c-bf56-e4d05da069c8 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:02.288051 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed13b6-48e2-4a5c-bf56-e4d05da069c8 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:02.288802 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed13b6-48e2-4a5c-bf56-e4d05da069c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:46:02.289110 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed13b6-48e2-4a5c-bf56-e4d05da069c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:02.290468 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed13b6-48e2-4a5c-bf56-e4d05da069c8 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:02.302429 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 537/1065] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:46:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:46:02.307798 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70750c2c-7ab5-4847-ae69-bc228598de36 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:02.403317 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70750c2c-7ab5-4847-ae69-bc228598de36 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:02.435219 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc3bb1b-d37e-44f3-9e33-e20b6d8e1fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:02.436003 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3bb1b-d37e-44f3-9e33-e20b6d8e1fcc None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:02.436287 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3bb1b-d37e-44f3-9e33-e20b6d8e1fcc None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:02.436565 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3bb1b-d37e-44f3-9e33-e20b6d8e1fcc None admin] PATH_INFO: `/v3/users/nova` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:02.445906 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc3bb1b-d37e-44f3-9e33-e20b6d8e1fcc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:02.447331 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc3bb1b-d37e-44f3-9e33-e20b6d8e1fcc None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:02.461413 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-cdc3bb1b-d37e-44f3-9e33-e20b6d8e1fcc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 06 14:46:02.461413 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:46:02.461413 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:46:02.461413 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:46:02.461413 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:46:02.461413 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:46:02.461413 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:46:02.461413 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:46:02.461413 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:46:02.461413 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:46:02.461413 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:46:02.461413 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:46:02.461413 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 06 14:46:02.461413 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_user(user_id) May 06 14:46:02.461413 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 06 14:46:02.461413 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 06 14:46:02.462378 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:46:02.462378 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:46:02.462378 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 06 14:46:02.462378 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:46:02.462378 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 06 14:46:02.462378 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:46:02.462378 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:46:02.462378 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:46:02.462378 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:46:02.462378 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:46:02.462378 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:46:02.462378 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:46:02.462378 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:46:02.462378 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:46:02.462378 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:46:02.462378 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:46:02.462378 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:46:02.462378 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:46:02.463367 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:46:02.463367 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:46:02.463367 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 06 14:46:02.463367 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 06 14:46:02.463367 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 06 14:46:02.463367 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 06 14:46:02.463367 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 06 14:46:02.463367 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 06 14:46:02.463367 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 06 14:46:02.463367 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:46:02.467322 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 529/1066] 10.222.0.23 () {56 vars in 1093 bytes} [Fri May 6 14:46:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:46:02.468297 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed13b6-48e2-4a5c-bf56-e4d05da069c8 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:02.572790 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed13b6-48e2-4a5c-bf56-e4d05da069c8 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:02.605679 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d89c50-cf0b-4c0a-86e8-129752ff479c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:02.606351 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d89c50-cf0b-4c0a-86e8-129752ff479c None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:02.606635 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d89c50-cf0b-4c0a-86e8-129752ff479c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:02.606911 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d89c50-cf0b-4c0a-86e8-129752ff479c None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:02.607746 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d89c50-cf0b-4c0a-86e8-129752ff479c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:46:02.608412 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d89c50-cf0b-4c0a-86e8-129752ff479c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:02.609880 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d89c50-cf0b-4c0a-86e8-129752ff479c None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:02.627544 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 538/1067] 10.222.0.23 () {56 vars in 1133 bytes} [Fri May 6 14:46:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:02.633486 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc3bb1b-d37e-44f3-9e33-e20b6d8e1fcc None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:02.730429 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc3bb1b-d37e-44f3-9e33-e20b6d8e1fcc None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:02.763545 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba2d62d-b293-43ed-9041-ccf924843d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:02.764337 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2d62d-b293-43ed-9041-ccf924843d4b None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:02.764632 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2d62d-b293-43ed-9041-ccf924843d4b None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:02.764916 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2d62d-b293-43ed-9041-ccf924843d4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:02.775480 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba2d62d-b293-43ed-9041-ccf924843d4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:02.776950 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba2d62d-b293-43ed-9041-ccf924843d4b None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:02.794481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-dba2d62d-b293-43ed-9041-ccf924843d4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:46:02.794481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:46:02.794481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:46:02.794481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:46:02.794481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:46:02.794481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:46:02.794481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:46:02.794481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:46:02.794481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:46:02.794481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:46:02.794481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:46:02.794481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:46:02.794481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:46:02.794481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:46:02.794481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:46:02.795388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:46:02.795388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:46:02.795388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:46:02.795388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:46:02.795388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:46:02.795388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:46:02.795388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:46:02.795388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:46:02.795388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:46:02.795388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:46:02.795388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:46:02.795388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:46:02.795388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:46:02.795388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:46:02.795388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:46:02.795388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:46:02.795388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:46:02.796301 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:46:02.796301 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:46:02.796301 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:46:02.796301 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:46:02.797147 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 530/1068] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:46:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:46:02.801858 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d89c50-cf0b-4c0a-86e8-129752ff479c None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:02.899061 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d89c50-cf0b-4c0a-86e8-129752ff479c None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:02.931950 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fee8b5-37e3-42bf-9185-29fdef8dfe44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:02.932637 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fee8b5-37e3-42bf-9185-29fdef8dfe44 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:02.932918 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fee8b5-37e3-42bf-9185-29fdef8dfe44 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:02.933191 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fee8b5-37e3-42bf-9185-29fdef8dfe44 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:02.933908 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fee8b5-37e3-42bf-9185-29fdef8dfe44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:46:02.934214 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fee8b5-37e3-42bf-9185-29fdef8dfe44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:02.935563 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fee8b5-37e3-42bf-9185-29fdef8dfe44 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:02.951096 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 539/1069] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:46:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:02.952264 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba2d62d-b293-43ed-9041-ccf924843d4b None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:03.053384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba2d62d-b293-43ed-9041-ccf924843d4b None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:03.086783 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e522da44-ef44-4b5b-b96c-fffe5c979c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:03.087573 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522da44-ef44-4b5b-b96c-fffe5c979c2e None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:03.087852 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522da44-ef44-4b5b-b96c-fffe5c979c2e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:03.088124 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522da44-ef44-4b5b-b96c-fffe5c979c2e None admin] PATH_INFO: `/v3/projects/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:03.094695 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e522da44-ef44-4b5b-b96c-fffe5c979c2e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:03.096115 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e522da44-ef44-4b5b-b96c-fffe5c979c2e None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:03.110097 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-e522da44-ef44-4b5b-b96c-fffe5c979c2e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:46:03.110097 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:46:03.110097 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:46:03.110097 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:46:03.110097 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:46:03.110097 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:46:03.110097 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:46:03.110097 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:46:03.110097 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:46:03.110097 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:46:03.110097 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:46:03.110097 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:46:03.110097 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 06 14:46:03.110097 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(project_id) May 06 14:46:03.110097 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 06 14:46:03.111025 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 06 14:46:03.111025 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:46:03.111025 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:46:03.111025 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:46:03.111025 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:46:03.111025 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:46:03.111025 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:46:03.111025 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:46:03.111025 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:46:03.111025 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:46:03.111025 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:46:03.111025 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:46:03.111025 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:46:03.111025 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:46:03.111025 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:46:03.111025 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:46:03.111025 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:46:03.117535 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 06 14:46:03.117535 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 06 14:46:03.117535 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 06 14:46:03.117535 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 06 14:46:03.117535 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 06 14:46:03.117535 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 06 14:46:03.117535 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:46:03.117535 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:46:03.117535 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 531/1070] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:46:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:46:03.120482 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fee8b5-37e3-42bf-9185-29fdef8dfe44 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:03.218505 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fee8b5-37e3-42bf-9185-29fdef8dfe44 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:03.219514 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494386c-cad7-459c-b6cc-1743c4c4915b None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:03.219796 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494386c-cad7-459c-b6cc-1743c4c4915b None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:03.220085 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494386c-cad7-459c-b6cc-1743c4c4915b None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:03.221482 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 532/1071] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:46:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:46:03.226756 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99579e2d-9c3c-4302-9ef3-59c5ace87425 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:03.227041 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99579e2d-9c3c-4302-9ef3-59c5ace87425 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:03.227318 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99579e2d-9c3c-4302-9ef3-59c5ace87425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:03.244528 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-99579e2d-9c3c-4302-9ef3-59c5ace87425 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:46:03.245738 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-99579e2d-9c3c-4302-9ef3-59c5ace87425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:46:03.257137 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10538c8-9d43-46e4-989c-1998053d054c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:03.258058 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10538c8-9d43-46e4-989c-1998053d054c None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:03.258339 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10538c8-9d43-46e4-989c-1998053d054c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:03.258613 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10538c8-9d43-46e4-989c-1998053d054c None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:03.259390 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10538c8-9d43-46e4-989c-1998053d054c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:46:03.259699 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10538c8-9d43-46e4-989c-1998053d054c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:03.261147 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10538c8-9d43-46e4-989c-1998053d054c None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:03.274302 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 540/1072] 10.222.0.23 () {56 vars in 1148 bytes} [Fri May 6 14:46:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:03.280311 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10538c8-9d43-46e4-989c-1998053d054c None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:03.314693 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 533/1073] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:46:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 06 14:46:03.337593 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99579e2d-9c3c-4302-9ef3-59c5ace87425 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:03.386137 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10538c8-9d43-46e4-989c-1998053d054c None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:03.420740 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e962853-177b-44a5-977f-dd357d8c91ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:03.421860 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e962853-177b-44a5-977f-dd357d8c91ba None admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:03.422293 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e962853-177b-44a5-977f-dd357d8c91ba None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:03.422743 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e962853-177b-44a5-977f-dd357d8c91ba None admin] PATH_INFO: `/v3/projects/d5398feee6eb490cb49bb261448a1a9b/users/fc8cf09642ec4e10a80badcf8b585ef8/roles/5213da7472024e87a41cecfaa37abce3` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:03.425177 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e962853-177b-44a5-977f-dd357d8c91ba None admin] RBAC: Authorizing `identity:create_grant(project_id=d5398feee6eb490cb49bb261448a1a9b, user_id=fc8cf09642ec4e10a80badcf8b585ef8, role_id=5213da7472024e87a41cecfaa37abce3)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:03.427035 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e962853-177b-44a5-977f-dd357d8c91ba None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:03.441464 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 541/1074] 10.222.0.23 () {58 vars in 1439 bytes} [Fri May 6 14:46:03 2022] PUT /identity/v3/projects/d5398feee6eb490cb49bb261448a1a9b/users/fc8cf09642ec4e10a80badcf8b585ef8/roles/5213da7472024e87a41cecfaa37abce3 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:46:03.448875 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99579e2d-9c3c-4302-9ef3-59c5ace87425 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:03.486326 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa8fade-06eb-4314-af00-2dd9e8a3da1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:03.487140 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa8fade-06eb-4314-af00-2dd9e8a3da1d None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:03.487451 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa8fade-06eb-4314-af00-2dd9e8a3da1d None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:03.487807 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa8fade-06eb-4314-af00-2dd9e8a3da1d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:03.499242 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa8fade-06eb-4314-af00-2dd9e8a3da1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:46:03.507975 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-ffa8fade-06eb-4314-af00-2dd9e8a3da1d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:46:03.507975 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:46:03.507975 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:46:03.507975 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:46:03.507975 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:46:03.507975 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:46:03.507975 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:46:03.507975 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:46:03.507975 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:46:03.507975 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:46:03.507975 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:46:03.507975 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:46:03.507975 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:46:03.507975 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:46:03.507975 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:46:03.507975 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:46:03.509034 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:46:03.509034 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:46:03.509034 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:46:03.509034 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:46:03.509034 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:46:03.509034 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:46:03.509034 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:46:03.509034 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:46:03.509034 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:46:03.509034 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:46:03.509034 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:46:03.509034 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:46:03.509034 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:46:03.509034 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:46:03.509034 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:46:03.509034 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:46:03.509941 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:46:03.509941 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:46:03.509941 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:46:03.509941 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:46:03.509941 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:46:03.509941 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:46:03.509941 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:46:03.509941 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:46:03.509941 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:46:03.509941 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:46:03.509941 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:46:03.509941 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:46:03.509941 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 534/1075] 10.222.0.23 () {56 vars in 1096 bytes} [Fri May 6 14:46:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:46:03.515533 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e962853-177b-44a5-977f-dd357d8c91ba None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:03.622743 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e962853-177b-44a5-977f-dd357d8c91ba None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:03.651154 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1796dcd-2228-4580-bb17-e104b8313b82 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:03.651460 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1796dcd-2228-4580-bb17-e104b8313b82 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:03.651721 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1796dcd-2228-4580-bb17-e104b8313b82 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:03.653196 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 535/1076] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:46:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:46:03.658793 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d928f-2cd5-4efb-a3ab-66647dac6858 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:03.658954 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30192772-088e-4571-8b8c-a1191203eab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:03.659057 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d928f-2cd5-4efb-a3ab-66647dac6858 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:03.659372 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d928f-2cd5-4efb-a3ab-66647dac6858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:03.659618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30192772-088e-4571-8b8c-a1191203eab6 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:03.659900 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30192772-088e-4571-8b8c-a1191203eab6 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:03.660335 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30192772-088e-4571-8b8c-a1191203eab6 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:03.661114 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30192772-088e-4571-8b8c-a1191203eab6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:46:03.661423 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30192772-088e-4571-8b8c-a1191203eab6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:03.662783 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30192772-088e-4571-8b8c-a1191203eab6 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:03.672634 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 542/1077] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:46:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:03.677265 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30192772-088e-4571-8b8c-a1191203eab6 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:03.677382 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-e44d928f-2cd5-4efb-a3ab-66647dac6858 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:46:03.678539 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-e44d928f-2cd5-4efb-a3ab-66647dac6858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:46:03.747357 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 536/1078] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:46:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 06 14:46:03.770217 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44d928f-2cd5-4efb-a3ab-66647dac6858 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:03.778022 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30192772-088e-4571-8b8c-a1191203eab6 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:03.815662 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee9531-5d98-4dba-ba0c-6150067e3c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:03.817100 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee9531-5d98-4dba-ba0c-6150067e3c6c None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:03.817382 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee9531-5d98-4dba-ba0c-6150067e3c6c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:03.817660 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee9531-5d98-4dba-ba0c-6150067e3c6c None admin] PATH_INFO: `/v3/users/d087368a067748179c11ed9d6e06711d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:03.818918 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee9531-5d98-4dba-ba0c-6150067e3c6c None admin] RBAC: Authorizing `identity:get_user(user_id=d087368a067748179c11ed9d6e06711d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:03.820343 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee9531-5d98-4dba-ba0c-6150067e3c6c None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:03.822169 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 543/1079] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:46:03 2022] GET /identity/v3/users/d087368a067748179c11ed9d6e06711d => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:46:03.827789 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee9531-5d98-4dba-ba0c-6150067e3c6c None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:03.879460 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44d928f-2cd5-4efb-a3ab-66647dac6858 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:03.914324 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8838a386-b58b-416f-b455-634b25eedb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:03.914979 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838a386-b58b-416f-b455-634b25eedb6b None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:03.915261 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838a386-b58b-416f-b455-634b25eedb6b None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:03.915535 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838a386-b58b-416f-b455-634b25eedb6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:03.924762 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee9531-5d98-4dba-ba0c-6150067e3c6c None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:03.958868 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c959f2a8-adb2-4666-b2df-283c97bdd2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:03.959732 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c959f2a8-adb2-4666-b2df-283c97bdd2c1 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:03.960038 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c959f2a8-adb2-4666-b2df-283c97bdd2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:03.960317 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c959f2a8-adb2-4666-b2df-283c97bdd2c1 None admin] PATH_INFO: `/v3/projects/4456b4addd3d46fa9e8410b04869c47e` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:03.961530 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c959f2a8-adb2-4666-b2df-283c97bdd2c1 None admin] RBAC: Authorizing `identity:get_project(project_id=4456b4addd3d46fa9e8410b04869c47e)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:03.963008 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c959f2a8-adb2-4666-b2df-283c97bdd2c1 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:03.964956 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 544/1080] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:46:03 2022] GET /identity/v3/projects/4456b4addd3d46fa9e8410b04869c47e => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:46:03.970405 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c959f2a8-adb2-4666-b2df-283c97bdd2c1 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:03.983137 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8838a386-b58b-416f-b455-634b25eedb6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:03.984587 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8838a386-b58b-416f-b455-634b25eedb6b None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:04.067388 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c959f2a8-adb2-4666-b2df-283c97bdd2c1 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:04.088366 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 537/1081] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:46:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 06 14:46:04.089445 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8838a386-b58b-416f-b455-634b25eedb6b None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:04.100710 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec82a0f-a251-4cb9-9e30-3e03275e5cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:04.101851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec82a0f-a251-4cb9-9e30-3e03275e5cf7 None admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:04.102338 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec82a0f-a251-4cb9-9e30-3e03275e5cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:04.102800 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec82a0f-a251-4cb9-9e30-3e03275e5cf7 None admin] PATH_INFO: `/v3/projects/4456b4addd3d46fa9e8410b04869c47e/users/d087368a067748179c11ed9d6e06711d/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:04.104940 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec82a0f-a251-4cb9-9e30-3e03275e5cf7 None admin] RBAC: Authorizing `identity:create_grant(project_id=4456b4addd3d46fa9e8410b04869c47e, user_id=d087368a067748179c11ed9d6e06711d, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:04.106355 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec82a0f-a251-4cb9-9e30-3e03275e5cf7 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:04.120251 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 545/1082] 10.222.0.23 () {58 vars in 1439 bytes} [Fri May 6 14:46:03 2022] PUT /identity/v3/projects/4456b4addd3d46fa9e8410b04869c47e/users/d087368a067748179c11ed9d6e06711d/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 14:46:04.195621 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8838a386-b58b-416f-b455-634b25eedb6b None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:04.232010 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a33986-fb49-4504-83f0-064d205bcecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:04.232855 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a33986-fb49-4504-83f0-064d205bcecf None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:04.233137 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a33986-fb49-4504-83f0-064d205bcecf None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:04.233409 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a33986-fb49-4504-83f0-064d205bcecf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:04.245180 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a33986-fb49-4504-83f0-064d205bcecf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:04.246649 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a33986-fb49-4504-83f0-064d205bcecf None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:04.266154 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-32a33986-fb49-4504-83f0-064d205bcecf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:46:04.266154 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:46:04.266154 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:46:04.266154 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:46:04.266154 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:46:04.266154 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:46:04.266154 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:46:04.266154 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:46:04.266154 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:46:04.266154 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:46:04.266154 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:46:04.266154 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:46:04.266154 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:46:04.266154 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:46:04.266154 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:46:04.267206 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:46:04.267206 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:46:04.267206 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:46:04.267206 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:46:04.267206 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:46:04.267206 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:46:04.267206 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:46:04.267206 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:46:04.267206 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:46:04.267206 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:46:04.267206 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:46:04.267206 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:46:04.267206 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:46:04.267206 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:46:04.267206 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:46:04.267206 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:46:04.267206 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:46:04.268162 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:46:04.268162 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:46:04.268162 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:46:04.268162 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:46:04.268162 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 538/1083] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:46:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:46:04.274321 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec82a0f-a251-4cb9-9e30-3e03275e5cf7 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:04.374327 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec82a0f-a251-4cb9-9e30-3e03275e5cf7 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:04.408034 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e3d8d-fb93-4cd5-b25f-d92bf1e720fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:04.409381 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e3d8d-fb93-4cd5-b25f-d92bf1e720fa None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:04.409672 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e3d8d-fb93-4cd5-b25f-d92bf1e720fa None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:04.409963 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e3d8d-fb93-4cd5-b25f-d92bf1e720fa None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:04.410734 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1e3d8d-fb93-4cd5-b25f-d92bf1e720fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:46:04.410992 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1e3d8d-fb93-4cd5-b25f-d92bf1e720fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:04.412353 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1e3d8d-fb93-4cd5-b25f-d92bf1e720fa None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:04.432702 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 546/1084] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:46:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:46:05.699937 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638373f5-292d-4805-a19b-841c9eb8de07 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:05.700468 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638373f5-292d-4805-a19b-841c9eb8de07 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:05.700468 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638373f5-292d-4805-a19b-841c9eb8de07 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:05.701855 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 539/1085] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:46:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:46:05.707356 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c6456-5223-4aa9-91d1-bf57d5fcc54c None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:05.707775 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c6456-5223-4aa9-91d1-bf57d5fcc54c None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:05.707941 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c6456-5223-4aa9-91d1-bf57d5fcc54c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:05.724173 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-792c6456-5223-4aa9-91d1-bf57d5fcc54c None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:46:05.725391 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-792c6456-5223-4aa9-91d1-bf57d5fcc54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:46:05.789034 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 547/1086] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:46:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:46:05.812733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638373f5-292d-4805-a19b-841c9eb8de07 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:05.913886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638373f5-292d-4805-a19b-841c9eb8de07 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:05.947192 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63e20a-b23a-49ac-9e6f-716c90dff97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:05.947990 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63e20a-b23a-49ac-9e6f-716c90dff97e None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:05.948269 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63e20a-b23a-49ac-9e6f-716c90dff97e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:05.948545 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63e20a-b23a-49ac-9e6f-716c90dff97e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:05.959330 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c63e20a-b23a-49ac-9e6f-716c90dff97e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:46:05.967338 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-3c63e20a-b23a-49ac-9e6f-716c90dff97e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 06 14:46:05.967338 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:46:05.967338 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:46:05.967338 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:46:05.967338 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:46:05.967338 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:46:05.967338 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:46:05.967338 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:46:05.967338 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:46:05.967338 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:46:05.967338 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:46:05.967338 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:46:05.967338 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:46:05.967338 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:46:05.967338 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:46:05.967338 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:46:05.968294 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:46:05.968294 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:46:05.968294 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:46:05.968294 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:46:05.968294 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:46:05.968294 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:46:05.968294 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:46:05.968294 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:46:05.968294 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:46:05.968294 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:46:05.968294 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:46:05.968294 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:46:05.968294 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:46:05.968294 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:46:05.968294 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:46:05.968294 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:46:05.969276 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:46:05.969276 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:46:05.969276 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:46:05.969276 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:46:05.969276 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:46:05.969276 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:46:05.969276 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:46:05.969276 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:46:05.969276 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:46:05.969276 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:46:05.969276 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 06 14:46:05.969276 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:46:05.969276 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 540/1087] 10.222.0.23 () {56 vars in 1120 bytes} [Fri May 6 14:46:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:46:05.974454 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c6456-5223-4aa9-91d1-bf57d5fcc54c None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:06.071696 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c6456-5223-4aa9-91d1-bf57d5fcc54c None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:06.104409 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a00910-cf49-4ff4-a3b2-e52b47322f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:06.105140 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a00910-cf49-4ff4-a3b2-e52b47322f1f None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:06.105421 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a00910-cf49-4ff4-a3b2-e52b47322f1f None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:06.105693 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a00910-cf49-4ff4-a3b2-e52b47322f1f None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:06.106418 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a00910-cf49-4ff4-a3b2-e52b47322f1f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:46:06.106716 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a00910-cf49-4ff4-a3b2-e52b47322f1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:06.108054 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a00910-cf49-4ff4-a3b2-e52b47322f1f None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:06.116517 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 548/1088] 10.222.0.23 () {56 vars in 1115 bytes} [Fri May 6 14:46:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:06.122405 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63e20a-b23a-49ac-9e6f-716c90dff97e None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:06.224537 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63e20a-b23a-49ac-9e6f-716c90dff97e None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:06.258121 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576d0acf-847f-444b-b589-28c711b11631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:06.258909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576d0acf-847f-444b-b589-28c711b11631 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:06.259190 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576d0acf-847f-444b-b589-28c711b11631 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:06.259465 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576d0acf-847f-444b-b589-28c711b11631 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:06.269731 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576d0acf-847f-444b-b589-28c711b11631 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:06.271149 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576d0acf-847f-444b-b589-28c711b11631 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:06.289071 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-576d0acf-847f-444b-b589-28c711b11631 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:46:06.289071 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:46:06.289071 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:46:06.289071 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:46:06.289071 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:46:06.289071 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:46:06.289071 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:46:06.289071 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:46:06.289071 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:46:06.289071 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:46:06.289071 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:46:06.289071 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:46:06.289071 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:46:06.289071 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:46:06.289071 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:46:06.290165 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:46:06.290165 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:46:06.290165 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:46:06.290165 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:46:06.290165 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:46:06.290165 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:46:06.290165 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:46:06.290165 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:46:06.290165 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:46:06.290165 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:46:06.290165 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:46:06.290165 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:46:06.290165 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:46:06.290165 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:46:06.290165 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:46:06.290165 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:46:06.290165 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:46:06.291093 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:46:06.291093 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:46:06.291093 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:46:06.291093 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:46:06.291093 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 541/1089] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:46:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:46:06.295984 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a00910-cf49-4ff4-a3b2-e52b47322f1f None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:06.360871 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600233c0-df8e-4de4-ac7a-a69d3cfffc7d None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:06.361154 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600233c0-df8e-4de4-ac7a-a69d3cfffc7d None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:06.361432 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600233c0-df8e-4de4-ac7a-a69d3cfffc7d None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:06.362875 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 542/1090] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:46:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:46:06.367962 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f18e98-1eac-4c0e-9818-10c7d18dcf17 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:06.368254 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f18e98-1eac-4c0e-9818-10c7d18dcf17 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:06.368532 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f18e98-1eac-4c0e-9818-10c7d18dcf17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:06.385244 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-84f18e98-1eac-4c0e-9818-10c7d18dcf17 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:46:06.386423 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-84f18e98-1eac-4c0e-9818-10c7d18dcf17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:46:06.392975 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a00910-cf49-4ff4-a3b2-e52b47322f1f None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:06.425239 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b983dce-7646-4806-8139-592f97561f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:06.425989 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b983dce-7646-4806-8139-592f97561f02 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:06.426270 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b983dce-7646-4806-8139-592f97561f02 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:06.426546 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b983dce-7646-4806-8139-592f97561f02 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:06.427250 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b983dce-7646-4806-8139-592f97561f02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:46:06.427559 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b983dce-7646-4806-8139-592f97561f02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:06.428939 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b983dce-7646-4806-8139-592f97561f02 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:06.445308 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 549/1091] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:46:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:06.446344 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b983dce-7646-4806-8139-592f97561f02 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:06.452965 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 543/1092] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:46:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 06 14:46:06.476841 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f18e98-1eac-4c0e-9818-10c7d18dcf17 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:06.544506 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b983dce-7646-4806-8139-592f97561f02 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:06.570521 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f18e98-1eac-4c0e-9818-10c7d18dcf17 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:06.578355 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de5cacf-299b-471e-ac5a-e2d1faf89124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:06.579142 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5cacf-299b-471e-ac5a-e2d1faf89124 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:06.579425 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5cacf-299b-471e-ac5a-e2d1faf89124 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:06.579695 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5cacf-299b-471e-ac5a-e2d1faf89124 None admin] PATH_INFO: `/v3/users/nova` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:06.587145 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de5cacf-299b-471e-ac5a-e2d1faf89124 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:06.588573 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de5cacf-299b-471e-ac5a-e2d1faf89124 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:06.602720 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-0de5cacf-299b-471e-ac5a-e2d1faf89124 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 06 14:46:06.602720 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:46:06.602720 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:46:06.602720 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:46:06.602720 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:46:06.602720 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:46:06.602720 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:46:06.602720 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:46:06.602720 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:46:06.602720 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:46:06.602720 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:46:06.602720 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:46:06.602720 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 06 14:46:06.602720 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_user(user_id) May 06 14:46:06.602720 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 06 14:46:06.602720 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 06 14:46:06.603899 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:46:06.603899 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:46:06.603899 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 06 14:46:06.603899 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:46:06.603899 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 06 14:46:06.603899 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:46:06.603899 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:46:06.603899 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:46:06.603899 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:46:06.603899 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:46:06.603899 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:46:06.603899 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:46:06.603899 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:46:06.603899 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:46:06.603899 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:46:06.603899 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:46:06.603899 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:46:06.603899 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:46:06.605119 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:46:06.605119 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:46:06.605119 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 06 14:46:06.605119 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 06 14:46:06.605119 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 06 14:46:06.605119 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 06 14:46:06.605119 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 06 14:46:06.605119 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 06 14:46:06.605119 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 06 14:46:06.605119 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:46:06.605119 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 550/1093] 10.222.0.23 () {56 vars in 1093 bytes} [Fri May 6 14:46:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:46:06.607569 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dd271c-a3e0-4d82-8ccf-bab9ef9c3b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:06.608360 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd271c-a3e0-4d82-8ccf-bab9ef9c3b8b None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:06.608662 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd271c-a3e0-4d82-8ccf-bab9ef9c3b8b None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:06.608939 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd271c-a3e0-4d82-8ccf-bab9ef9c3b8b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:06.609584 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de5cacf-299b-471e-ac5a-e2d1faf89124 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:06.619572 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dd271c-a3e0-4d82-8ccf-bab9ef9c3b8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:46:06.627105 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-b6dd271c-a3e0-4d82-8ccf-bab9ef9c3b8b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:46:06.627105 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:46:06.627105 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:46:06.627105 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:46:06.627105 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:46:06.627105 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:46:06.627105 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:46:06.627105 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:46:06.627105 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:46:06.627105 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:46:06.627105 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:46:06.627105 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:46:06.627105 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:46:06.627105 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:46:06.627105 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:46:06.627105 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:46:06.628703 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:46:06.628703 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:46:06.628703 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:46:06.628703 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:46:06.628703 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:46:06.628703 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:46:06.628703 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:46:06.628703 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:46:06.628703 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:46:06.628703 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:46:06.628703 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:46:06.628703 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:46:06.628703 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:46:06.628703 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:46:06.628703 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:46:06.628703 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:46:06.629568 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:46:06.629568 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:46:06.629568 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:46:06.629568 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:46:06.629568 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:46:06.629568 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:46:06.629568 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:46:06.629568 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:46:06.629568 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:46:06.629568 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:46:06.629568 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:46:06.629568 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:46:06.629568 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 544/1094] 10.222.0.23 () {56 vars in 1096 bytes} [Fri May 6 14:46:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:46:06.635069 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dd271c-a3e0-4d82-8ccf-bab9ef9c3b8b None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:06.704105 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de5cacf-299b-471e-ac5a-e2d1faf89124 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:06.730804 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dd271c-a3e0-4d82-8ccf-bab9ef9c3b8b None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:06.737959 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2c85d-941b-41af-8448-6b5d5cb7bab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:06.739209 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2c85d-941b-41af-8448-6b5d5cb7bab3 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:06.739502 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2c85d-941b-41af-8448-6b5d5cb7bab3 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:06.739787 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2c85d-941b-41af-8448-6b5d5cb7bab3 None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:06.740644 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee2c85d-941b-41af-8448-6b5d5cb7bab3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:46:06.740961 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee2c85d-941b-41af-8448-6b5d5cb7bab3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:06.742388 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee2c85d-941b-41af-8448-6b5d5cb7bab3 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:06.759455 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 551/1095] 10.222.0.23 () {56 vars in 1133 bytes} [Fri May 6 14:46:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:06.763746 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c990e75-c884-45eb-b420-7110cc933eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:06.764410 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c990e75-c884-45eb-b420-7110cc933eb7 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:06.764717 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c990e75-c884-45eb-b420-7110cc933eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:06.764995 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c990e75-c884-45eb-b420-7110cc933eb7 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:06.765253 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2c85d-941b-41af-8448-6b5d5cb7bab3 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:06.765732 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c990e75-c884-45eb-b420-7110cc933eb7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:46:06.766035 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c990e75-c884-45eb-b420-7110cc933eb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:06.767392 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c990e75-c884-45eb-b420-7110cc933eb7 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:06.775401 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 545/1096] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:46:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:46:06.781241 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c990e75-c884-45eb-b420-7110cc933eb7 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:06.859968 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2c85d-941b-41af-8448-6b5d5cb7bab3 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:06.875112 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c990e75-c884-45eb-b420-7110cc933eb7 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:06.892458 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145dbcb-85f7-4c86-a627-61b93f793ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:06.893131 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145dbcb-85f7-4c86-a627-61b93f793ec5 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:06.893425 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145dbcb-85f7-4c86-a627-61b93f793ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:06.894318 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145dbcb-85f7-4c86-a627-61b93f793ec5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:06.907148 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd1e48d-da4b-494e-9cd2-afc3e9002df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:06.908008 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd1e48d-da4b-494e-9cd2-afc3e9002df7 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:06.908286 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd1e48d-da4b-494e-9cd2-afc3e9002df7 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:06.908568 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd1e48d-da4b-494e-9cd2-afc3e9002df7 None admin] PATH_INFO: `/v3/users/d087368a067748179c11ed9d6e06711d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:06.910436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd1e48d-da4b-494e-9cd2-afc3e9002df7 None admin] RBAC: Authorizing `identity:get_user(user_id=d087368a067748179c11ed9d6e06711d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:06.911866 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd1e48d-da4b-494e-9cd2-afc3e9002df7 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:06.918296 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 546/1097] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:46:06 2022] GET /identity/v3/users/d087368a067748179c11ed9d6e06711d => generated 285 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:46:06.919063 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd1e48d-da4b-494e-9cd2-afc3e9002df7 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:06.958625 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e145dbcb-85f7-4c86-a627-61b93f793ec5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:06.960060 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e145dbcb-85f7-4c86-a627-61b93f793ec5 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:07.012161 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd1e48d-da4b-494e-9cd2-afc3e9002df7 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:07.045736 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f14c13b-eb1f-4b1d-b266-af0cbaa12313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:07.046376 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14c13b-eb1f-4b1d-b266-af0cbaa12313 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:07.046659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14c13b-eb1f-4b1d-b266-af0cbaa12313 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:07.046929 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14c13b-eb1f-4b1d-b266-af0cbaa12313 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:07.054385 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 552/1098] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:46:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 06 14:46:07.060067 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145dbcb-85f7-4c86-a627-61b93f793ec5 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:07.110960 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f14c13b-eb1f-4b1d-b266-af0cbaa12313 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:07.112464 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f14c13b-eb1f-4b1d-b266-af0cbaa12313 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:07.153306 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145dbcb-85f7-4c86-a627-61b93f793ec5 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:07.187043 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa45243-9430-4b94-b378-8bfc0c0fea28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:07.187834 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa45243-9430-4b94-b378-8bfc0c0fea28 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:07.188114 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa45243-9430-4b94-b378-8bfc0c0fea28 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:07.188389 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa45243-9430-4b94-b378-8bfc0c0fea28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:07.198784 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa45243-9430-4b94-b378-8bfc0c0fea28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:07.200214 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa45243-9430-4b94-b378-8bfc0c0fea28 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:07.208367 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 547/1099] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:46:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 06 14:46:07.214128 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f14c13b-eb1f-4b1d-b266-af0cbaa12313 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:07.217843 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-aaa45243-9430-4b94-b378-8bfc0c0fea28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:46:07.217843 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:46:07.217843 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:46:07.217843 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:46:07.217843 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:46:07.217843 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:46:07.217843 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:46:07.217843 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:46:07.217843 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:46:07.217843 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:46:07.217843 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:46:07.217843 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:46:07.217843 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:46:07.217843 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:46:07.217843 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:46:07.218957 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:46:07.218957 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:46:07.218957 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:46:07.218957 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:46:07.218957 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:46:07.218957 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:46:07.218957 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:46:07.218957 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:46:07.218957 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:46:07.218957 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:46:07.218957 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:46:07.218957 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:46:07.218957 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:46:07.218957 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:46:07.218957 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:46:07.218957 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:46:07.218957 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:46:07.219864 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:46:07.219864 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:46:07.219864 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:46:07.219864 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:46:07.219864 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 553/1100] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:46:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:46:07.224430 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa45243-9430-4b94-b378-8bfc0c0fea28 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:07.309813 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f14c13b-eb1f-4b1d-b266-af0cbaa12313 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:07.318596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa45243-9430-4b94-b378-8bfc0c0fea28 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:07.341355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd11df8-9817-4798-aa70-1e6658d4255d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:07.342207 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd11df8-9817-4798-aa70-1e6658d4255d None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:07.342488 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd11df8-9817-4798-aa70-1e6658d4255d None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:07.342760 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd11df8-9817-4798-aa70-1e6658d4255d None admin] PATH_INFO: `/v3/projects/4456b4addd3d46fa9e8410b04869c47e` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:07.343920 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd11df8-9817-4798-aa70-1e6658d4255d None admin] RBAC: Authorizing `identity:get_project(project_id=4456b4addd3d46fa9e8410b04869c47e)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:07.345364 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd11df8-9817-4798-aa70-1e6658d4255d None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:07.347966 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 548/1101] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:46:07 2022] GET /identity/v3/projects/4456b4addd3d46fa9e8410b04869c47e => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:46:07.351037 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f4a834-9b0e-4861-a856-39c72230323f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:07.351763 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4a834-9b0e-4861-a856-39c72230323f None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:07.352113 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4a834-9b0e-4861-a856-39c72230323f None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:07.352455 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4a834-9b0e-4861-a856-39c72230323f None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:07.353462 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f4a834-9b0e-4861-a856-39c72230323f None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:46:07.353640 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd11df8-9817-4798-aa70-1e6658d4255d None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:07.353882 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f4a834-9b0e-4861-a856-39c72230323f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:07.355233 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f4a834-9b0e-4861-a856-39c72230323f None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:07.366948 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 554/1102] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:46:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:46:07.372370 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f4a834-9b0e-4861-a856-39c72230323f None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:07.449754 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd11df8-9817-4798-aa70-1e6658d4255d None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:07.470259 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f4a834-9b0e-4861-a856-39c72230323f None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:07.482204 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3bb562-f3e9-4dea-a60b-6a01c3104caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:07.482863 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3bb562-f3e9-4dea-a60b-6a01c3104caf None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:07.483146 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3bb562-f3e9-4dea-a60b-6a01c3104caf None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:07.483418 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3bb562-f3e9-4dea-a60b-6a01c3104caf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:07.484309 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3bb562-f3e9-4dea-a60b-6a01c3104caf None admin] RBAC: Adding query filter params (role.id=6dcd35d1a83748759d6ef9f026077835, scope.project.id=4456b4addd3d46fa9e8410b04869c47e, user.id=d087368a067748179c11ed9d6e06711d) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:46:07.484646 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3bb562-f3e9-4dea-a60b-6a01c3104caf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:07.486069 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3bb562-f3e9-4dea-a60b-6a01c3104caf None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:07.492305 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 549/1103] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:46:07 2022] GET /identity/v3/role_assignments?user.id=d087368a067748179c11ed9d6e06711d&scope.project.id=4456b4addd3d46fa9e8410b04869c47e&role.id=6dcd35d1a83748759d6ef9f026077835 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:46:07.504526 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f37009-2208-402f-b025-892dca261cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:07.505381 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f37009-2208-402f-b025-892dca261cc7 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:07.505669 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f37009-2208-402f-b025-892dca261cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:07.505947 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f37009-2208-402f-b025-892dca261cc7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:07.513733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f37009-2208-402f-b025-892dca261cc7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:07.515200 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f37009-2208-402f-b025-892dca261cc7 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:07.531639 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-e0f37009-2208-402f-b025-892dca261cc7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:46:07.531639 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:46:07.531639 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:46:07.531639 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:46:07.531639 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:46:07.531639 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:46:07.531639 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:46:07.531639 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:46:07.531639 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:46:07.531639 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:46:07.531639 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:46:07.531639 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:46:07.531639 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 06 14:46:07.531639 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(project_id) May 06 14:46:07.531639 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 06 14:46:07.532771 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 06 14:46:07.532771 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:46:07.532771 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:46:07.532771 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:46:07.532771 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:46:07.532771 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:46:07.532771 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:46:07.532771 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:46:07.532771 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:46:07.532771 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:46:07.532771 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:46:07.532771 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:46:07.532771 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:46:07.532771 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:46:07.532771 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:46:07.532771 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:46:07.532771 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:46:07.533682 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 06 14:46:07.533682 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 06 14:46:07.533682 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 06 14:46:07.533682 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 06 14:46:07.533682 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 06 14:46:07.533682 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 06 14:46:07.533682 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:46:07.533682 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:46:07.533682 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 555/1104] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:46:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:46:07.539708 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3bb562-f3e9-4dea-a60b-6a01c3104caf None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:07.642218 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3bb562-f3e9-4dea-a60b-6a01c3104caf None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:07.678000 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4777e1-5112-48e4-9fb0-c1b44821e814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:07.678662 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4777e1-5112-48e4-9fb0-c1b44821e814 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:07.679676 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4777e1-5112-48e4-9fb0-c1b44821e814 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:07.679676 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4777e1-5112-48e4-9fb0-c1b44821e814 None admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:07.680689 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4777e1-5112-48e4-9fb0-c1b44821e814 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:46:07.680689 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4777e1-5112-48e4-9fb0-c1b44821e814 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:07.681786 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4777e1-5112-48e4-9fb0-c1b44821e814 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:07.693794 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 550/1105] 10.222.0.23 () {56 vars in 1148 bytes} [Fri May 6 14:46:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:07.699771 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f37009-2208-402f-b025-892dca261cc7 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:07.796371 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f37009-2208-402f-b025-892dca261cc7 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:07.828271 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe3ca0-b1e5-4266-930e-0ccb7424db75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:07.828947 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe3ca0-b1e5-4266-930e-0ccb7424db75 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:07.829231 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe3ca0-b1e5-4266-930e-0ccb7424db75 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:07.829503 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe3ca0-b1e5-4266-930e-0ccb7424db75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:07.830318 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fe3ca0-b1e5-4266-930e-0ccb7424db75 None admin] RBAC: Adding query filter params (role.id=5213da7472024e87a41cecfaa37abce3, scope.project.id=d5398feee6eb490cb49bb261448a1a9b, user.id=fc8cf09642ec4e10a80badcf8b585ef8) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:46:07.830626 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fe3ca0-b1e5-4266-930e-0ccb7424db75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:07.832614 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fe3ca0-b1e5-4266-930e-0ccb7424db75 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:07.838262 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 556/1106] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:46:07 2022] GET /identity/v3/role_assignments?user.id=fc8cf09642ec4e10a80badcf8b585ef8&scope.project.id=d5398feee6eb490cb49bb261448a1a9b&role.id=5213da7472024e87a41cecfaa37abce3 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:09.777305 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f878150a-6837-4674-bcd4-93f6a0fe83f8 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:09.777828 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f878150a-6837-4674-bcd4-93f6a0fe83f8 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:09.777943 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f878150a-6837-4674-bcd4-93f6a0fe83f8 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:09.779192 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 551/1107] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:46:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:46:09.785141 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548ee96-6b0a-44db-ad4c-8316eedf000b None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:09.785564 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548ee96-6b0a-44db-ad4c-8316eedf000b None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:09.785694 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548ee96-6b0a-44db-ad4c-8316eedf000b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:09.802465 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-a548ee96-6b0a-44db-ad4c-8316eedf000b None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:46:09.803622 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-a548ee96-6b0a-44db-ad4c-8316eedf000b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:46:09.866043 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 557/1108] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:46:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:46:09.890848 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f878150a-6837-4674-bcd4-93f6a0fe83f8 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:09.986871 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f878150a-6837-4674-bcd4-93f6a0fe83f8 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:10.020847 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003ca7b-aed4-444f-a7a7-95b4673117c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:10.021623 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003ca7b-aed4-444f-a7a7-95b4673117c3 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:10.021907 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003ca7b-aed4-444f-a7a7-95b4673117c3 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:10.022186 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003ca7b-aed4-444f-a7a7-95b4673117c3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:10.023342 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b003ca7b-aed4-444f-a7a7-95b4673117c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:10.024783 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b003ca7b-aed4-444f-a7a7-95b4673117c3 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:10.026363 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 552/1109] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:46:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:10.032452 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a548ee96-6b0a-44db-ad4c-8316eedf000b None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:10.125932 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a548ee96-6b0a-44db-ad4c-8316eedf000b None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:10.158923 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f00522-07b8-4046-ba50-04299dd64979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:10.160196 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f00522-07b8-4046-ba50-04299dd64979 None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:10.160557 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f00522-07b8-4046-ba50-04299dd64979 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:10.160789 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f00522-07b8-4046-ba50-04299dd64979 None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:10.161440 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f00522-07b8-4046-ba50-04299dd64979 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:10.162871 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f00522-07b8-4046-ba50-04299dd64979 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:10.182598 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 558/1110] 10.222.0.23 () {60 vars in 1132 bytes} [Fri May 6 14:46:10 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 14:46:12.439860 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ea7ff-947e-4289-b75c-543accd3347c None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:12.441053 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ea7ff-947e-4289-b75c-543accd3347c None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:12.441411 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ea7ff-947e-4289-b75c-543accd3347c None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:12.442757 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 553/1111] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:46:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:46:12.448021 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340232f-9cbe-440e-82e2-8d74be576ef4 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:12.448400 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340232f-9cbe-440e-82e2-8d74be576ef4 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:12.448692 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340232f-9cbe-440e-82e2-8d74be576ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:12.465773 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-9340232f-9cbe-440e-82e2-8d74be576ef4 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:46:12.467097 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-9340232f-9cbe-440e-82e2-8d74be576ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:46:12.534291 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 559/1112] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:46:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:46:12.556979 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57ea7ff-947e-4289-b75c-543accd3347c None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:12.654584 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57ea7ff-947e-4289-b75c-543accd3347c None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:12.686407 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e9811-bf7a-420e-9aff-e8c1761f0160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:12.687247 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e9811-bf7a-420e-9aff-e8c1761f0160 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:12.687525 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e9811-bf7a-420e-9aff-e8c1761f0160 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:12.687795 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e9811-bf7a-420e-9aff-e8c1761f0160 None admin] PATH_INFO: `/v3/roles/d328f8040f2247d083309d839d2ff947` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:12.689259 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e9811-bf7a-420e-9aff-e8c1761f0160 None admin] RBAC: Authorizing `identity:get_role(role_id=d328f8040f2247d083309d839d2ff947)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:12.691319 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e9811-bf7a-420e-9aff-e8c1761f0160 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:12.692589 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 554/1113] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:46:12 2022] GET /identity/v3/roles/d328f8040f2247d083309d839d2ff947 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:12.698320 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9340232f-9cbe-440e-82e2-8d74be576ef4 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:12.793439 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9340232f-9cbe-440e-82e2-8d74be576ef4 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:12.826772 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24479ade-fa63-4cf5-a0bd-400316c6d9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:12.827671 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24479ade-fa63-4cf5-a0bd-400316c6d9a9 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:12.827956 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24479ade-fa63-4cf5-a0bd-400316c6d9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:12.828238 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24479ade-fa63-4cf5-a0bd-400316c6d9a9 None admin] PATH_INFO: `/v3/users/9a04eff1309249138f59365a49409e23` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:12.840998 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24479ade-fa63-4cf5-a0bd-400316c6d9a9 None admin] RBAC: Authorizing `identity:get_user(user_id=9a04eff1309249138f59365a49409e23)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:12.842434 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24479ade-fa63-4cf5-a0bd-400316c6d9a9 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:12.844099 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 560/1114] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:46:12 2022] GET /identity/v3/users/9a04eff1309249138f59365a49409e23 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:12.850107 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e9811-bf7a-420e-9aff-e8c1761f0160 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:12.945200 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e9811-bf7a-420e-9aff-e8c1761f0160 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:12.976864 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c844ec60-2d12-4ab8-ac19-c5c5e9271c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:12.977505 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c844ec60-2d12-4ab8-ac19-c5c5e9271c2a None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:12.977788 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c844ec60-2d12-4ab8-ac19-c5c5e9271c2a None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:12.978067 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c844ec60-2d12-4ab8-ac19-c5c5e9271c2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:13.041142 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c844ec60-2d12-4ab8-ac19-c5c5e9271c2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:13.042573 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c844ec60-2d12-4ab8-ac19-c5c5e9271c2a None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:13.135158 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 555/1115] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:46:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 06 14:46:13.141099 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24479ade-fa63-4cf5-a0bd-400316c6d9a9 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:13.235067 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24479ade-fa63-4cf5-a0bd-400316c6d9a9 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:13.266573 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b286e-b7a6-4bc4-a87c-cfd4ba63e6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:13.267414 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b286e-b7a6-4bc4-a87c-cfd4ba63e6b1 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:13.267691 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b286e-b7a6-4bc4-a87c-cfd4ba63e6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:13.267960 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b286e-b7a6-4bc4-a87c-cfd4ba63e6b1 None admin] PATH_INFO: `/v3/projects/4456b4addd3d46fa9e8410b04869c47e` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:13.269395 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0b286e-b7a6-4bc4-a87c-cfd4ba63e6b1 None admin] RBAC: Authorizing `identity:get_project(project_id=4456b4addd3d46fa9e8410b04869c47e)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:13.270802 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0b286e-b7a6-4bc4-a87c-cfd4ba63e6b1 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:13.272468 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 561/1116] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:46:13 2022] GET /identity/v3/projects/4456b4addd3d46fa9e8410b04869c47e => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:13.278583 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c844ec60-2d12-4ab8-ac19-c5c5e9271c2a None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:13.372001 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c844ec60-2d12-4ab8-ac19-c5c5e9271c2a None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:13.403430 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e09b5-13f2-41c0-9614-e89a0e4837f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:13.404069 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e09b5-13f2-41c0-9614-e89a0e4837f3 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:13.404352 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e09b5-13f2-41c0-9614-e89a0e4837f3 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:13.404651 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e09b5-13f2-41c0-9614-e89a0e4837f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:13.405732 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635e09b5-13f2-41c0-9614-e89a0e4837f3 None admin] RBAC: Adding query filter params (role.id=d328f8040f2247d083309d839d2ff947, scope.project.id=4456b4addd3d46fa9e8410b04869c47e, user.id=9a04eff1309249138f59365a49409e23) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:46:13.406051 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635e09b5-13f2-41c0-9614-e89a0e4837f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:13.407466 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635e09b5-13f2-41c0-9614-e89a0e4837f3 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:13.413459 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 556/1117] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:46:13 2022] GET /identity/v3/role_assignments?user.id=9a04eff1309249138f59365a49409e23&scope.project.id=4456b4addd3d46fa9e8410b04869c47e&role.id=d328f8040f2247d083309d839d2ff947 => generated 259 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:15.619808 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e40626b-5d8e-4799-8b60-b4acdcefbdf0 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:15.620368 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e40626b-5d8e-4799-8b60-b4acdcefbdf0 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:15.620368 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e40626b-5d8e-4799-8b60-b4acdcefbdf0 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:15.621926 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 562/1118] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:46:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:46:15.627234 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a14822-bbed-40fd-9cba-266c294d32c9 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:15.628182 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a14822-bbed-40fd-9cba-266c294d32c9 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:15.628406 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a14822-bbed-40fd-9cba-266c294d32c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:15.645167 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-02a14822-bbed-40fd-9cba-266c294d32c9 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:46:15.646345 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-02a14822-bbed-40fd-9cba-266c294d32c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:46:15.709826 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 557/1119] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:46:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:46:15.732506 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e40626b-5d8e-4799-8b60-b4acdcefbdf0 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:15.828582 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e40626b-5d8e-4799-8b60-b4acdcefbdf0 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:15.861305 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c3666-3b71-4457-817c-7cb0a1426f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:15.862173 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c3666-3b71-4457-817c-7cb0a1426f0b None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:15.862452 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c3666-3b71-4457-817c-7cb0a1426f0b None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:15.862723 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c3666-3b71-4457-817c-7cb0a1426f0b None admin] PATH_INFO: `/v3/roles/d328f8040f2247d083309d839d2ff947` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:15.864178 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58c3666-3b71-4457-817c-7cb0a1426f0b None admin] RBAC: Authorizing `identity:get_role(role_id=d328f8040f2247d083309d839d2ff947)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:15.865560 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58c3666-3b71-4457-817c-7cb0a1426f0b None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:15.866916 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 563/1120] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:46:15 2022] GET /identity/v3/roles/d328f8040f2247d083309d839d2ff947 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:15.872924 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a14822-bbed-40fd-9cba-266c294d32c9 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:15.968649 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a14822-bbed-40fd-9cba-266c294d32c9 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:16.000725 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1425eddd-d163-43c7-8bd7-fd0d4d67d667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:16.001581 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425eddd-d163-43c7-8bd7-fd0d4d67d667 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:16.001862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425eddd-d163-43c7-8bd7-fd0d4d67d667 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:16.002133 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425eddd-d163-43c7-8bd7-fd0d4d67d667 None admin] PATH_INFO: `/v3/users/9a04eff1309249138f59365a49409e23` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:16.003369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1425eddd-d163-43c7-8bd7-fd0d4d67d667 None admin] RBAC: Authorizing `identity:get_user(user_id=9a04eff1309249138f59365a49409e23)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:16.004807 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1425eddd-d163-43c7-8bd7-fd0d4d67d667 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:16.006397 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 558/1121] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:46:15 2022] GET /identity/v3/users/9a04eff1309249138f59365a49409e23 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:16.012179 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c3666-3b71-4457-817c-7cb0a1426f0b None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:16.106576 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c3666-3b71-4457-817c-7cb0a1426f0b None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:16.138406 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77057bf-223e-4c49-bc00-85974f663d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:16.139176 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77057bf-223e-4c49-bc00-85974f663d99 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:16.139461 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77057bf-223e-4c49-bc00-85974f663d99 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:16.139738 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77057bf-223e-4c49-bc00-85974f663d99 None admin] PATH_INFO: `/v3/projects/4456b4addd3d46fa9e8410b04869c47e` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:16.140918 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77057bf-223e-4c49-bc00-85974f663d99 None admin] RBAC: Authorizing `identity:get_project(project_id=4456b4addd3d46fa9e8410b04869c47e)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:16.142335 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77057bf-223e-4c49-bc00-85974f663d99 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:16.144625 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 564/1122] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:46:16 2022] GET /identity/v3/projects/4456b4addd3d46fa9e8410b04869c47e => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:46:16.150500 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1425eddd-d163-43c7-8bd7-fd0d4d67d667 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:16.243507 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1425eddd-d163-43c7-8bd7-fd0d4d67d667 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:16.275870 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e40d20-9b49-4bfa-a056-e578c3c60c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:16.276768 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e40d20-9b49-4bfa-a056-e578c3c60c56 None admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:16.277047 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e40d20-9b49-4bfa-a056-e578c3c60c56 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:16.277315 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e40d20-9b49-4bfa-a056-e578c3c60c56 None admin] PATH_INFO: `/v3/projects/4456b4addd3d46fa9e8410b04869c47e/users/9a04eff1309249138f59365a49409e23/roles/d328f8040f2247d083309d839d2ff947` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:16.279459 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e40d20-9b49-4bfa-a056-e578c3c60c56 None admin] RBAC: Authorizing `identity:create_grant(project_id=4456b4addd3d46fa9e8410b04869c47e, user_id=9a04eff1309249138f59365a49409e23, role_id=d328f8040f2247d083309d839d2ff947)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:16.281173 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e40d20-9b49-4bfa-a056-e578c3c60c56 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:16.293237 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 559/1123] 10.222.0.23 () {58 vars in 1439 bytes} [Fri May 6 14:46:16 2022] PUT /identity/v3/projects/4456b4addd3d46fa9e8410b04869c47e/users/9a04eff1309249138f59365a49409e23/roles/d328f8040f2247d083309d839d2ff947 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:46:18.517614 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55196ea-b0cd-4634-bd87-1c6be9e82b38 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:18.518714 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55196ea-b0cd-4634-bd87-1c6be9e82b38 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:18.518714 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55196ea-b0cd-4634-bd87-1c6be9e82b38 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:18.520071 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 565/1124] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:46:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:46:18.525343 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e56a5e-b634-46a4-a2e3-38fc9727ca95 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:18.525763 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e56a5e-b634-46a4-a2e3-38fc9727ca95 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:18.525908 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e56a5e-b634-46a4-a2e3-38fc9727ca95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:18.542865 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-d9e56a5e-b634-46a4-a2e3-38fc9727ca95 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:46:18.544032 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d9e56a5e-b634-46a4-a2e3-38fc9727ca95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:46:18.606032 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 560/1125] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:46:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 06 14:46:18.630254 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55196ea-b0cd-4634-bd87-1c6be9e82b38 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:18.727033 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55196ea-b0cd-4634-bd87-1c6be9e82b38 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:18.759294 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107007e3-1a8f-493f-ac60-01e39eb91f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:18.760133 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107007e3-1a8f-493f-ac60-01e39eb91f48 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:18.760415 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107007e3-1a8f-493f-ac60-01e39eb91f48 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:18.760721 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107007e3-1a8f-493f-ac60-01e39eb91f48 None admin] PATH_INFO: `/v3/roles/d328f8040f2247d083309d839d2ff947` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:18.762170 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107007e3-1a8f-493f-ac60-01e39eb91f48 None admin] RBAC: Authorizing `identity:get_role(role_id=d328f8040f2247d083309d839d2ff947)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:18.763524 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107007e3-1a8f-493f-ac60-01e39eb91f48 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:18.764897 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 566/1126] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:46:18 2022] GET /identity/v3/roles/d328f8040f2247d083309d839d2ff947 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:18.771272 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e56a5e-b634-46a4-a2e3-38fc9727ca95 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:18.865214 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e56a5e-b634-46a4-a2e3-38fc9727ca95 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:18.897437 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab0f013-f765-4d0c-8a09-23e135710655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:18.898331 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab0f013-f765-4d0c-8a09-23e135710655 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:18.898613 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab0f013-f765-4d0c-8a09-23e135710655 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:18.898887 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab0f013-f765-4d0c-8a09-23e135710655 None admin] PATH_INFO: `/v3/users/9a04eff1309249138f59365a49409e23` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:18.900077 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab0f013-f765-4d0c-8a09-23e135710655 None admin] RBAC: Authorizing `identity:get_user(user_id=9a04eff1309249138f59365a49409e23)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:18.901528 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab0f013-f765-4d0c-8a09-23e135710655 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:18.903149 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 561/1127] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:46:18 2022] GET /identity/v3/users/9a04eff1309249138f59365a49409e23 => generated 286 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:18.909748 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107007e3-1a8f-493f-ac60-01e39eb91f48 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:19.003156 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107007e3-1a8f-493f-ac60-01e39eb91f48 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:19.035981 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794074cc-9c6a-45f2-b5e0-e6bd715bbfb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:19.036638 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794074cc-9c6a-45f2-b5e0-e6bd715bbfb9 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:19.036924 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794074cc-9c6a-45f2-b5e0-e6bd715bbfb9 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:19.037204 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794074cc-9c6a-45f2-b5e0-e6bd715bbfb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:19.100581 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794074cc-9c6a-45f2-b5e0-e6bd715bbfb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:19.102082 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794074cc-9c6a-45f2-b5e0-e6bd715bbfb9 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:19.196365 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 567/1128] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:46:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 06 14:46:19.202243 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab0f013-f765-4d0c-8a09-23e135710655 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:19.299055 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab0f013-f765-4d0c-8a09-23e135710655 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:19.332042 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a197e1ae-45d0-44f9-aec9-71fdf9470f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:19.332917 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197e1ae-45d0-44f9-aec9-71fdf9470f7b None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:19.333200 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197e1ae-45d0-44f9-aec9-71fdf9470f7b None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:19.333476 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197e1ae-45d0-44f9-aec9-71fdf9470f7b None admin] PATH_INFO: `/v3/projects/4456b4addd3d46fa9e8410b04869c47e` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:19.334639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a197e1ae-45d0-44f9-aec9-71fdf9470f7b None admin] RBAC: Authorizing `identity:get_project(project_id=4456b4addd3d46fa9e8410b04869c47e)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:19.336119 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a197e1ae-45d0-44f9-aec9-71fdf9470f7b None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:19.337847 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 562/1129] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:46:19 2022] GET /identity/v3/projects/4456b4addd3d46fa9e8410b04869c47e => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:19.343944 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794074cc-9c6a-45f2-b5e0-e6bd715bbfb9 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:19.436817 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794074cc-9c6a-45f2-b5e0-e6bd715bbfb9 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:19.469630 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b1b24-9d7f-4886-a682-dc736a215230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:19.470274 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b1b24-9d7f-4886-a682-dc736a215230 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:19.470560 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b1b24-9d7f-4886-a682-dc736a215230 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:19.470838 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b1b24-9d7f-4886-a682-dc736a215230 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:19.471716 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b1b24-9d7f-4886-a682-dc736a215230 None admin] RBAC: Adding query filter params (role.id=d328f8040f2247d083309d839d2ff947, scope.project.id=4456b4addd3d46fa9e8410b04869c47e, user.id=9a04eff1309249138f59365a49409e23) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:46:19.472024 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b1b24-9d7f-4886-a682-dc736a215230 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:19.473470 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b1b24-9d7f-4886-a682-dc736a215230 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:19.479042 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 568/1130] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:46:19 2022] GET /identity/v3/role_assignments?user.id=9a04eff1309249138f59365a49409e23&scope.project.id=4456b4addd3d46fa9e8410b04869c47e&role.id=d328f8040f2247d083309d839d2ff947 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:21.732025 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f5744-f649-49b5-98d6-7d3ebac18dca None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:21.732565 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f5744-f649-49b5-98d6-7d3ebac18dca None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:21.732565 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f5744-f649-49b5-98d6-7d3ebac18dca None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:21.733930 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 563/1131] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:46:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:46:21.739821 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e706a356-9a8b-4939-9653-3d17c607bd39 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:21.740253 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e706a356-9a8b-4939-9653-3d17c607bd39 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:21.740389 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e706a356-9a8b-4939-9653-3d17c607bd39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:21.757189 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-e706a356-9a8b-4939-9653-3d17c607bd39 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:46:21.758372 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e706a356-9a8b-4939-9653-3d17c607bd39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:46:21.822590 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 569/1132] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:46:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:46:21.847727 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f5744-f649-49b5-98d6-7d3ebac18dca None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:21.946704 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f5744-f649-49b5-98d6-7d3ebac18dca None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:21.979711 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98051bf4-0be2-493c-9fa7-f187377ea94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:21.980488 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98051bf4-0be2-493c-9fa7-f187377ea94f None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:21.980797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98051bf4-0be2-493c-9fa7-f187377ea94f None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:21.981074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98051bf4-0be2-493c-9fa7-f187377ea94f None admin] PATH_INFO: `/v3/domains/default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:21.982279 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98051bf4-0be2-493c-9fa7-f187377ea94f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:21.983651 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98051bf4-0be2-493c-9fa7-f187377ea94f None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:21.985276 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 564/1133] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:46:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:21.991928 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e706a356-9a8b-4939-9653-3d17c607bd39 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:22.090064 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e706a356-9a8b-4939-9653-3d17c607bd39 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:22.122271 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804eacf1-af91-43ba-a2b6-cb2616793f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:22.122907 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804eacf1-af91-43ba-a2b6-cb2616793f95 None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:22.123183 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804eacf1-af91-43ba-a2b6-cb2616793f95 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:22.123453 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804eacf1-af91-43ba-a2b6-cb2616793f95 None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:22.124135 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804eacf1-af91-43ba-a2b6-cb2616793f95 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:22.125563 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804eacf1-af91-43ba-a2b6-cb2616793f95 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:22.151291 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 570/1134] 10.222.0.23 () {60 vars in 1141 bytes} [Fri May 6 14:46:21 2022] POST /identity/v3/projects => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:46:24.544061 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f2beb-deac-4259-b739-324a6a161dda None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:24.544627 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f2beb-deac-4259-b739-324a6a161dda None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:24.544627 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f2beb-deac-4259-b739-324a6a161dda None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:24.545997 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 565/1135] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:46:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:46:24.551426 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f205c-1362-4b99-93ed-ee0e3d54d835 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:24.551800 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f205c-1362-4b99-93ed-ee0e3d54d835 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:24.551988 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f205c-1362-4b99-93ed-ee0e3d54d835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:24.568851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-c63f205c-1362-4b99-93ed-ee0e3d54d835 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:46:24.570025 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-c63f205c-1362-4b99-93ed-ee0e3d54d835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:46:24.633251 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 571/1136] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:46:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:46:24.658474 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f2beb-deac-4259-b739-324a6a161dda None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:24.756395 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f2beb-deac-4259-b739-324a6a161dda None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:24.788281 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6294889-4c59-4ada-a422-7f039390c09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:24.789079 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6294889-4c59-4ada-a422-7f039390c09e None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:24.789359 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6294889-4c59-4ada-a422-7f039390c09e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:24.789631 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6294889-4c59-4ada-a422-7f039390c09e None admin] PATH_INFO: `/v3/domains/default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:24.790843 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6294889-4c59-4ada-a422-7f039390c09e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:24.792262 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6294889-4c59-4ada-a422-7f039390c09e None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:24.793858 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 566/1137] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:46:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:24.799609 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f205c-1362-4b99-93ed-ee0e3d54d835 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:24.892560 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f205c-1362-4b99-93ed-ee0e3d54d835 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:24.925701 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e6ae5e-f3e4-47fb-9d10-21363a0750c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:24.926350 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e6ae5e-f3e4-47fb-9d10-21363a0750c0 None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:24.926631 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e6ae5e-f3e4-47fb-9d10-21363a0750c0 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:24.926905 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e6ae5e-f3e4-47fb-9d10-21363a0750c0 None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:24.927604 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e6ae5e-f3e4-47fb-9d10-21363a0750c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:24.929092 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e6ae5e-f3e4-47fb-9d10-21363a0750c0 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:24.949055 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 572/1138] 10.222.0.23 () {60 vars in 1132 bytes} [Fri May 6 14:46:24 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:46:27.190569 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ca4bc-5809-4163-99f8-f39cdcea037c None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:27.191066 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ca4bc-5809-4163-99f8-f39cdcea037c None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:27.191158 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ca4bc-5809-4163-99f8-f39cdcea037c None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:27.192472 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 567/1139] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:46:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:46:27.198288 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1616f363-e745-40a1-bb9b-832b96cc1b7c None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:27.198681 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1616f363-e745-40a1-bb9b-832b96cc1b7c None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:27.198835 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1616f363-e745-40a1-bb9b-832b96cc1b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:27.215919 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-1616f363-e745-40a1-bb9b-832b96cc1b7c None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:46:27.217110 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-1616f363-e745-40a1-bb9b-832b96cc1b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:46:27.281863 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 573/1140] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:46:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 06 14:46:27.305798 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ca4bc-5809-4163-99f8-f39cdcea037c None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:27.401420 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ca4bc-5809-4163-99f8-f39cdcea037c None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:27.433142 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe2dbd-523a-4fb0-ac20-7d9a6ec70680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:27.433857 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe2dbd-523a-4fb0-ac20-7d9a6ec70680 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:27.434136 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe2dbd-523a-4fb0-ac20-7d9a6ec70680 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:27.434414 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe2dbd-523a-4fb0-ac20-7d9a6ec70680 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:27.444629 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fe2dbd-523a-4fb0-ac20-7d9a6ec70680 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:46:27.452146 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-14fe2dbd-523a-4fb0-ac20-7d9a6ec70680 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:46:27.452146 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:46:27.452146 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:46:27.452146 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:46:27.452146 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:46:27.452146 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:46:27.452146 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:46:27.452146 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:46:27.452146 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:46:27.452146 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:46:27.452146 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:46:27.452146 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:46:27.452146 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:46:27.452146 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:46:27.452146 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:46:27.452146 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:46:27.453275 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:46:27.453275 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:46:27.453275 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:46:27.453275 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:46:27.453275 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:46:27.453275 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:46:27.453275 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:46:27.453275 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:46:27.453275 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:46:27.453275 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:46:27.453275 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:46:27.453275 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:46:27.453275 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:46:27.453275 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:46:27.453275 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:46:27.453275 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:46:27.454329 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:46:27.454329 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:46:27.454329 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:46:27.454329 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:46:27.454329 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:46:27.454329 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:46:27.454329 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:46:27.454329 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:46:27.454329 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:46:27.454329 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:46:27.454329 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:46:27.454329 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:46:27.454329 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 568/1141] 10.222.0.23 () {56 vars in 1096 bytes} [Fri May 6 14:46:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:46:27.459985 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1616f363-e745-40a1-bb9b-832b96cc1b7c None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:27.556243 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1616f363-e745-40a1-bb9b-832b96cc1b7c None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:27.588947 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c274e658-dd3c-43bc-8580-de7a658aa420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:27.589666 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c274e658-dd3c-43bc-8580-de7a658aa420 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:27.590057 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c274e658-dd3c-43bc-8580-de7a658aa420 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:27.590337 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c274e658-dd3c-43bc-8580-de7a658aa420 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:27.591197 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c274e658-dd3c-43bc-8580-de7a658aa420 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:46:27.591413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c274e658-dd3c-43bc-8580-de7a658aa420 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:27.592828 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c274e658-dd3c-43bc-8580-de7a658aa420 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:27.601373 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 574/1142] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:46:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:27.607244 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe2dbd-523a-4fb0-ac20-7d9a6ec70680 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:27.703219 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe2dbd-523a-4fb0-ac20-7d9a6ec70680 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:27.735332 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05826475-1802-4006-afb8-c28eddb81258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:27.736174 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05826475-1802-4006-afb8-c28eddb81258 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:27.736448 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05826475-1802-4006-afb8-c28eddb81258 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:27.736914 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05826475-1802-4006-afb8-c28eddb81258 None admin] PATH_INFO: `/v3/users/45f0d6574ed2453fa91f17240e9a0f13` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:27.750296 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05826475-1802-4006-afb8-c28eddb81258 None admin] RBAC: Authorizing `identity:get_user(user_id=45f0d6574ed2453fa91f17240e9a0f13)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:27.751722 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05826475-1802-4006-afb8-c28eddb81258 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:27.754121 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 569/1143] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:46:27 2022] GET /identity/v3/users/45f0d6574ed2453fa91f17240e9a0f13 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:27.759945 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c274e658-dd3c-43bc-8580-de7a658aa420 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:27.853875 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c274e658-dd3c-43bc-8580-de7a658aa420 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:27.888057 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defde350-ed6c-40d4-8d15-b5644c7a2d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:27.888719 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defde350-ed6c-40d4-8d15-b5644c7a2d91 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:27.888999 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defde350-ed6c-40d4-8d15-b5644c7a2d91 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:27.889271 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defde350-ed6c-40d4-8d15-b5644c7a2d91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:27.952961 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defde350-ed6c-40d4-8d15-b5644c7a2d91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:27.954422 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defde350-ed6c-40d4-8d15-b5644c7a2d91 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:28.046461 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 575/1144] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:46:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 06 14:46:28.053238 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05826475-1802-4006-afb8-c28eddb81258 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:28.148383 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05826475-1802-4006-afb8-c28eddb81258 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:28.181947 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72318d46-2296-4522-b396-3454a6baefd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:28.182795 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72318d46-2296-4522-b396-3454a6baefd8 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:28.183078 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72318d46-2296-4522-b396-3454a6baefd8 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:28.183353 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72318d46-2296-4522-b396-3454a6baefd8 None admin] PATH_INFO: `/v3/projects/26df05655bb046d4889e559543a14a5a` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:28.184506 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72318d46-2296-4522-b396-3454a6baefd8 None admin] RBAC: Authorizing `identity:get_project(project_id=26df05655bb046d4889e559543a14a5a)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:28.185951 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72318d46-2296-4522-b396-3454a6baefd8 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:28.187615 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 570/1145] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:46:28 2022] GET /identity/v3/projects/26df05655bb046d4889e559543a14a5a => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:28.194265 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defde350-ed6c-40d4-8d15-b5644c7a2d91 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:28.289056 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defde350-ed6c-40d4-8d15-b5644c7a2d91 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:28.321558 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2816d3ef-9fd6-4aca-958c-9bcff84d8a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:28.322201 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816d3ef-9fd6-4aca-958c-9bcff84d8a36 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:28.322487 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816d3ef-9fd6-4aca-958c-9bcff84d8a36 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:28.322764 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816d3ef-9fd6-4aca-958c-9bcff84d8a36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:28.323595 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2816d3ef-9fd6-4aca-958c-9bcff84d8a36 None admin] RBAC: Adding query filter params (role.id=6dcd35d1a83748759d6ef9f026077835, scope.project.id=26df05655bb046d4889e559543a14a5a, user.id=45f0d6574ed2453fa91f17240e9a0f13) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:46:28.323890 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2816d3ef-9fd6-4aca-958c-9bcff84d8a36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:28.325392 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2816d3ef-9fd6-4aca-958c-9bcff84d8a36 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:28.330608 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 576/1146] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:46:28 2022] GET /identity/v3/role_assignments?user.id=45f0d6574ed2453fa91f17240e9a0f13&scope.project.id=26df05655bb046d4889e559543a14a5a&role.id=6dcd35d1a83748759d6ef9f026077835 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:30.535836 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d897cd-80e3-40d7-947c-11004488b076 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:30.536376 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d897cd-80e3-40d7-947c-11004488b076 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:30.536472 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d897cd-80e3-40d7-947c-11004488b076 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:30.537791 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 571/1147] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:46:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:46:30.543368 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01e42f-89c1-4846-be10-145c9a227bc5 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:30.543810 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01e42f-89c1-4846-be10-145c9a227bc5 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:30.543932 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01e42f-89c1-4846-be10-145c9a227bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:30.560931 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-5d01e42f-89c1-4846-be10-145c9a227bc5 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:46:30.562101 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-5d01e42f-89c1-4846-be10-145c9a227bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:46:30.624133 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 577/1148] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:46:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:46:30.647628 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d897cd-80e3-40d7-947c-11004488b076 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:30.743927 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d897cd-80e3-40d7-947c-11004488b076 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:30.775782 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5935cb1-1e3b-4be2-a6d7-0944591e939d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:30.776568 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5935cb1-1e3b-4be2-a6d7-0944591e939d None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:30.776934 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5935cb1-1e3b-4be2-a6d7-0944591e939d None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:30.777153 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5935cb1-1e3b-4be2-a6d7-0944591e939d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:30.788087 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5935cb1-1e3b-4be2-a6d7-0944591e939d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:46:30.796196 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-f5935cb1-1e3b-4be2-a6d7-0944591e939d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:46:30.796196 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:46:30.796196 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:46:30.796196 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:46:30.796196 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:46:30.796196 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:46:30.796196 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:46:30.796196 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:46:30.796196 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:46:30.796196 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:46:30.796196 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:46:30.796196 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:46:30.796196 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:46:30.796196 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:46:30.796196 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:46:30.796196 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:46:30.797371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:46:30.797371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:46:30.797371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:46:30.797371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:46:30.797371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:46:30.797371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:46:30.797371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:46:30.797371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:46:30.797371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:46:30.797371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:46:30.797371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:46:30.797371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:46:30.797371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:46:30.797371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:46:30.797371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:46:30.797371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:46:30.798436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:46:30.798436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:46:30.798436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:46:30.798436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:46:30.798436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:46:30.798436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:46:30.798436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:46:30.798436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:46:30.798436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:46:30.798436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:46:30.798436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:46:30.798436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:46:30.798436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 572/1149] 10.222.0.23 () {56 vars in 1096 bytes} [Fri May 6 14:46:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:46:30.804152 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d01e42f-89c1-4846-be10-145c9a227bc5 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:30.898957 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d01e42f-89c1-4846-be10-145c9a227bc5 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:30.931800 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb771b5-80ff-4da8-a79d-1a426b12384d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:30.932452 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb771b5-80ff-4da8-a79d-1a426b12384d None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:30.932756 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb771b5-80ff-4da8-a79d-1a426b12384d None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:30.933032 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb771b5-80ff-4da8-a79d-1a426b12384d None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:30.933753 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb771b5-80ff-4da8-a79d-1a426b12384d None admin] RBAC: Adding query filter params (name=admin) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:46:30.934055 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb771b5-80ff-4da8-a79d-1a426b12384d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:30.935390 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb771b5-80ff-4da8-a79d-1a426b12384d None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:30.943994 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 578/1150] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:46:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:30.949784 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5935cb1-1e3b-4be2-a6d7-0944591e939d None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:31.044901 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5935cb1-1e3b-4be2-a6d7-0944591e939d None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:31.077700 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f992a5-fb04-414a-bea6-30889c8391ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:31.078544 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f992a5-fb04-414a-bea6-30889c8391ab None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:31.078820 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f992a5-fb04-414a-bea6-30889c8391ab None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:31.079090 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f992a5-fb04-414a-bea6-30889c8391ab None admin] PATH_INFO: `/v3/users/45f0d6574ed2453fa91f17240e9a0f13` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:31.080233 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f992a5-fb04-414a-bea6-30889c8391ab None admin] RBAC: Authorizing `identity:get_user(user_id=45f0d6574ed2453fa91f17240e9a0f13)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:31.082105 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f992a5-fb04-414a-bea6-30889c8391ab None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:31.083689 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 573/1151] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:46:30 2022] GET /identity/v3/users/45f0d6574ed2453fa91f17240e9a0f13 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:31.089348 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb771b5-80ff-4da8-a79d-1a426b12384d None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:31.185752 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb771b5-80ff-4da8-a79d-1a426b12384d None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:31.216884 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8fea87-e969-4fe9-bcc3-575ccb46346d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:31.217741 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8fea87-e969-4fe9-bcc3-575ccb46346d None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:31.218020 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8fea87-e969-4fe9-bcc3-575ccb46346d None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:31.218293 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8fea87-e969-4fe9-bcc3-575ccb46346d None admin] PATH_INFO: `/v3/projects/26df05655bb046d4889e559543a14a5a` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:31.219445 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8fea87-e969-4fe9-bcc3-575ccb46346d None admin] RBAC: Authorizing `identity:get_project(project_id=26df05655bb046d4889e559543a14a5a)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:31.220895 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8fea87-e969-4fe9-bcc3-575ccb46346d None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:31.223109 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 579/1152] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:46:31 2022] GET /identity/v3/projects/26df05655bb046d4889e559543a14a5a => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:31.228972 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f992a5-fb04-414a-bea6-30889c8391ab None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:31.323148 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f992a5-fb04-414a-bea6-30889c8391ab None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:31.354565 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b18922-0768-49c8-8dcb-bfdd040e935a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:31.355426 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b18922-0768-49c8-8dcb-bfdd040e935a None admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:31.355705 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b18922-0768-49c8-8dcb-bfdd040e935a None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:31.355971 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b18922-0768-49c8-8dcb-bfdd040e935a None admin] PATH_INFO: `/v3/projects/26df05655bb046d4889e559543a14a5a/users/45f0d6574ed2453fa91f17240e9a0f13/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:31.357897 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b18922-0768-49c8-8dcb-bfdd040e935a None admin] RBAC: Authorizing `identity:create_grant(project_id=26df05655bb046d4889e559543a14a5a, user_id=45f0d6574ed2453fa91f17240e9a0f13, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:31.359301 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b18922-0768-49c8-8dcb-bfdd040e935a None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:31.372312 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 574/1153] 10.222.0.23 () {58 vars in 1439 bytes} [Fri May 6 14:46:31 2022] PUT /identity/v3/projects/26df05655bb046d4889e559543a14a5a/users/45f0d6574ed2453fa91f17240e9a0f13/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:46:33.621954 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87930d-68f1-4dec-97ea-3f12c4eea60a None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:33.622486 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87930d-68f1-4dec-97ea-3f12c4eea60a None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:33.622486 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87930d-68f1-4dec-97ea-3f12c4eea60a None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:33.623843 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 580/1154] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:46:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:46:33.629812 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63216046-d58c-44ff-a472-fa3846982ee4 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:33.630231 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63216046-d58c-44ff-a472-fa3846982ee4 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:33.630380 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63216046-d58c-44ff-a472-fa3846982ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:33.648952 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-63216046-d58c-44ff-a472-fa3846982ee4 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:46:33.650183 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-63216046-d58c-44ff-a472-fa3846982ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:46:33.721066 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 575/1155] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:46:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 06 14:46:33.744846 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d87930d-68f1-4dec-97ea-3f12c4eea60a None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:33.842055 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d87930d-68f1-4dec-97ea-3f12c4eea60a None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:33.873571 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a6e81-819c-4f29-821a-2b3233eb7880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:33.874285 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a6e81-819c-4f29-821a-2b3233eb7880 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:33.874566 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a6e81-819c-4f29-821a-2b3233eb7880 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:33.874838 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a6e81-819c-4f29-821a-2b3233eb7880 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:33.885133 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53a6e81-819c-4f29-821a-2b3233eb7880 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:46:33.893591 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-b53a6e81-819c-4f29-821a-2b3233eb7880 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:46:33.893591 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:46:33.893591 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:46:33.893591 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:46:33.893591 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:46:33.893591 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:46:33.893591 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:46:33.893591 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:46:33.893591 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:46:33.893591 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:46:33.893591 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:46:33.893591 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:46:33.893591 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:46:33.893591 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:46:33.893591 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:46:33.893591 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:46:33.894801 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:46:33.894801 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:46:33.894801 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:46:33.894801 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:46:33.894801 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:46:33.894801 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:46:33.894801 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:46:33.894801 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:46:33.894801 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:46:33.894801 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:46:33.894801 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:46:33.894801 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:46:33.894801 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:46:33.894801 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:46:33.894801 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:46:33.894801 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:46:33.895858 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:46:33.895858 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:46:33.895858 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:46:33.895858 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:46:33.895858 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:46:33.895858 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:46:33.895858 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:46:33.895858 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:46:33.895858 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:46:33.895858 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:46:33.895858 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:46:33.895858 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:46:33.895858 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 581/1156] 10.222.0.23 () {56 vars in 1096 bytes} [Fri May 6 14:46:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:46:33.900875 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63216046-d58c-44ff-a472-fa3846982ee4 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:33.999953 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63216046-d58c-44ff-a472-fa3846982ee4 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:34.032574 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b55898-2777-4173-9904-1ac2e7f719f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:34.033288 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b55898-2777-4173-9904-1ac2e7f719f2 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:34.033569 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b55898-2777-4173-9904-1ac2e7f719f2 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:34.035075 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b55898-2777-4173-9904-1ac2e7f719f2 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:34.035860 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b55898-2777-4173-9904-1ac2e7f719f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:46:34.036172 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b55898-2777-4173-9904-1ac2e7f719f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:34.037602 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b55898-2777-4173-9904-1ac2e7f719f2 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:34.045372 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 576/1157] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:46:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:34.051311 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a6e81-819c-4f29-821a-2b3233eb7880 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:34.147541 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a6e81-819c-4f29-821a-2b3233eb7880 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:34.179684 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377b9275-29e5-4a55-9b31-4f23cbca75b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:34.180542 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b9275-29e5-4a55-9b31-4f23cbca75b3 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:34.180881 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b9275-29e5-4a55-9b31-4f23cbca75b3 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:34.181158 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b9275-29e5-4a55-9b31-4f23cbca75b3 None admin] PATH_INFO: `/v3/users/45f0d6574ed2453fa91f17240e9a0f13` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:34.182333 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377b9275-29e5-4a55-9b31-4f23cbca75b3 None admin] RBAC: Authorizing `identity:get_user(user_id=45f0d6574ed2453fa91f17240e9a0f13)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:34.183754 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377b9275-29e5-4a55-9b31-4f23cbca75b3 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:34.185528 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 582/1158] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:46:34 2022] GET /identity/v3/users/45f0d6574ed2453fa91f17240e9a0f13 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:34.192317 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b55898-2777-4173-9904-1ac2e7f719f2 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:34.289954 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b55898-2777-4173-9904-1ac2e7f719f2 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:34.325690 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f9f41f-5865-41cf-be6a-c1e53ae16090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:34.326323 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9f41f-5865-41cf-be6a-c1e53ae16090 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:34.326603 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9f41f-5865-41cf-be6a-c1e53ae16090 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:34.326874 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9f41f-5865-41cf-be6a-c1e53ae16090 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:34.390690 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f9f41f-5865-41cf-be6a-c1e53ae16090 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:34.392126 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f9f41f-5865-41cf-be6a-c1e53ae16090 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:34.485281 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 577/1159] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:46:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 06 14:46:34.491876 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377b9275-29e5-4a55-9b31-4f23cbca75b3 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:34.586868 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377b9275-29e5-4a55-9b31-4f23cbca75b3 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:34.618528 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc28b5f0-b551-444a-9020-bee9a914d3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:34.619370 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28b5f0-b551-444a-9020-bee9a914d3e5 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:34.619651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28b5f0-b551-444a-9020-bee9a914d3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:34.619925 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28b5f0-b551-444a-9020-bee9a914d3e5 None admin] PATH_INFO: `/v3/projects/26df05655bb046d4889e559543a14a5a` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:34.621103 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc28b5f0-b551-444a-9020-bee9a914d3e5 None admin] RBAC: Authorizing `identity:get_project(project_id=26df05655bb046d4889e559543a14a5a)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:34.622540 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc28b5f0-b551-444a-9020-bee9a914d3e5 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:34.624217 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 583/1160] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:46:34 2022] GET /identity/v3/projects/26df05655bb046d4889e559543a14a5a => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:34.630994 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f9f41f-5865-41cf-be6a-c1e53ae16090 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:34.725124 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f9f41f-5865-41cf-be6a-c1e53ae16090 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:34.757512 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c2c180-0109-45d7-bdec-8ed452aa8d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:34.758152 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c2c180-0109-45d7-bdec-8ed452aa8d54 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:34.758435 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c2c180-0109-45d7-bdec-8ed452aa8d54 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:34.758707 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c2c180-0109-45d7-bdec-8ed452aa8d54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:34.759532 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c2c180-0109-45d7-bdec-8ed452aa8d54 None admin] RBAC: Adding query filter params (role.id=6dcd35d1a83748759d6ef9f026077835, scope.project.id=26df05655bb046d4889e559543a14a5a, user.id=45f0d6574ed2453fa91f17240e9a0f13) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:46:34.759839 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c2c180-0109-45d7-bdec-8ed452aa8d54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:34.761297 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c2c180-0109-45d7-bdec-8ed452aa8d54 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:34.767428 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 578/1161] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:46:34 2022] GET /identity/v3/role_assignments?user.id=45f0d6574ed2453fa91f17240e9a0f13&scope.project.id=26df05655bb046d4889e559543a14a5a&role.id=6dcd35d1a83748759d6ef9f026077835 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:37.007286 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708b4d0-9e8e-4f5f-958e-310038a561a3 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:37.007847 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708b4d0-9e8e-4f5f-958e-310038a561a3 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:37.007847 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708b4d0-9e8e-4f5f-958e-310038a561a3 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:37.009261 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 584/1162] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:46:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:46:37.014525 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a168485-62fc-43a4-ba74-a21f92ebe49a None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:37.014957 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a168485-62fc-43a4-ba74-a21f92ebe49a None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:37.015062 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a168485-62fc-43a4-ba74-a21f92ebe49a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:37.032441 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-3a168485-62fc-43a4-ba74-a21f92ebe49a None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:46:37.034531 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-3a168485-62fc-43a4-ba74-a21f92ebe49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:46:37.097646 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 579/1163] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:46:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:46:37.119783 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708b4d0-9e8e-4f5f-958e-310038a561a3 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:37.217696 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708b4d0-9e8e-4f5f-958e-310038a561a3 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:37.249390 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95b707-1c8a-4405-ae2d-7e85daf4be89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:37.250035 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95b707-1c8a-4405-ae2d-7e85daf4be89 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:37.250316 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95b707-1c8a-4405-ae2d-7e85daf4be89 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:37.250592 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95b707-1c8a-4405-ae2d-7e85daf4be89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:37.314577 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc95b707-1c8a-4405-ae2d-7e85daf4be89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:37.316592 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc95b707-1c8a-4405-ae2d-7e85daf4be89 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:37.408443 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 585/1164] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:46:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 06 14:46:37.414602 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a168485-62fc-43a4-ba74-a21f92ebe49a None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:37.509040 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a168485-62fc-43a4-ba74-a21f92ebe49a None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:37.541399 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521b6210-4ef7-4445-b4e4-962d238c8251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:37.542195 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b6210-4ef7-4445-b4e4-962d238c8251 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:37.542474 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b6210-4ef7-4445-b4e4-962d238c8251 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:37.542746 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b6210-4ef7-4445-b4e4-962d238c8251 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:37.549737 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521b6210-4ef7-4445-b4e4-962d238c8251 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:37.551299 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521b6210-4ef7-4445-b4e4-962d238c8251 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:37.565976 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-521b6210-4ef7-4445-b4e4-962d238c8251 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 06 14:46:37.565976 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:46:37.565976 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 06 14:46:37.565976 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 06 14:46:37.565976 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 06 14:46:37.565976 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 06 14:46:37.565976 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 06 14:46:37.565976 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 06 14:46:37.565976 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 06 14:46:37.565976 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:46:37.565976 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 06 14:46:37.565976 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:46:37.565976 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:46:37.565976 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:46:37.565976 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:46:37.565976 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:46:37.565976 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:46:37.567224 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:46:37.567224 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:46:37.567224 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:46:37.567224 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:46:37.567224 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:46:37.567224 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:46:37.567224 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:46:37.567224 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:46:37.567224 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:46:37.567224 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:46:37.567224 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:46:37.567224 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:46:37.567224 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:46:37.567224 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:46:37.567224 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:46:37.567224 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:46:37.568479 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:46:37.568479 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:46:37.568479 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:46:37.568479 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:46:37.568479 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:46:37.568479 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:46:37.568479 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:46:37.568479 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:46:37.568479 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:46:37.568479 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:46:37.568479 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 06 14:46:37.568479 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:46:37.568479 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 06 14:46:37.568479 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:46:37.568479 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 580/1165] 10.222.0.23 () {56 vars in 1117 bytes} [Fri May 6 14:46:37 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:46:37.573108 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95b707-1c8a-4405-ae2d-7e85daf4be89 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:37.666463 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95b707-1c8a-4405-ae2d-7e85daf4be89 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:37.700249 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60de31ce-1a39-4a5d-a4a6-57ffdd7600c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:37.701413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60de31ce-1a39-4a5d-a4a6-57ffdd7600c1 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:37.701939 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60de31ce-1a39-4a5d-a4a6-57ffdd7600c1 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:37.702467 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60de31ce-1a39-4a5d-a4a6-57ffdd7600c1 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:37.703806 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60de31ce-1a39-4a5d-a4a6-57ffdd7600c1 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:46:37.704371 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60de31ce-1a39-4a5d-a4a6-57ffdd7600c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:37.707004 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60de31ce-1a39-4a5d-a4a6-57ffdd7600c1 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:37.717490 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 586/1166] 10.222.0.23 () {56 vars in 1115 bytes} [Fri May 6 14:46:37 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:39.932691 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d658b7-1da5-487c-a5e0-019faacdf01d None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:39.933314 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d658b7-1da5-487c-a5e0-019faacdf01d None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:39.933314 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d658b7-1da5-487c-a5e0-019faacdf01d None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:39.934579 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 581/1167] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:46:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:46:39.939804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a90b4b-fcfb-49d0-b336-62aaec68c871 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:39.940203 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a90b4b-fcfb-49d0-b336-62aaec68c871 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:39.940358 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a90b4b-fcfb-49d0-b336-62aaec68c871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:39.957339 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-b9a90b4b-fcfb-49d0-b336-62aaec68c871 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:46:39.958511 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-b9a90b4b-fcfb-49d0-b336-62aaec68c871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:46:40.022528 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 587/1168] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:46:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:46:40.046017 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d658b7-1da5-487c-a5e0-019faacdf01d None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:40.144320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d658b7-1da5-487c-a5e0-019faacdf01d None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:40.177748 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75728f0e-b19f-4927-89b8-69f994ba5bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:40.178403 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75728f0e-b19f-4927-89b8-69f994ba5bdb None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:40.178686 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75728f0e-b19f-4927-89b8-69f994ba5bdb None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:40.178963 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75728f0e-b19f-4927-89b8-69f994ba5bdb None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:40.179689 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75728f0e-b19f-4927-89b8-69f994ba5bdb None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:40.181086 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75728f0e-b19f-4927-89b8-69f994ba5bdb None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:40.195674 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 582/1169] 10.222.0.23 () {60 vars in 1138 bytes} [Fri May 6 14:46:40 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:46:42.464455 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0982cd8a-7bd6-4267-86cb-cca31917bd28 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:42.464977 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0982cd8a-7bd6-4267-86cb-cca31917bd28 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:42.464977 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0982cd8a-7bd6-4267-86cb-cca31917bd28 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:42.466334 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 588/1170] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:46:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:46:42.471687 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5fa2d0-f5ec-4710-9898-ceb84b0820ee None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:42.472092 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5fa2d0-f5ec-4710-9898-ceb84b0820ee None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:42.472237 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5fa2d0-f5ec-4710-9898-ceb84b0820ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:42.488428 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-bd5fa2d0-f5ec-4710-9898-ceb84b0820ee None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:46:42.489631 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-bd5fa2d0-f5ec-4710-9898-ceb84b0820ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:46:42.552868 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 583/1171] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:46:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:46:42.576456 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0982cd8a-7bd6-4267-86cb-cca31917bd28 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:42.672957 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0982cd8a-7bd6-4267-86cb-cca31917bd28 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:42.705715 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6565cadc-79c5-40fe-be09-4f4cc56ddff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:42.706489 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565cadc-79c5-40fe-be09-4f4cc56ddff2 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:42.706771 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565cadc-79c5-40fe-be09-4f4cc56ddff2 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:42.707049 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565cadc-79c5-40fe-be09-4f4cc56ddff2 None admin] PATH_INFO: `/v3/domains/6aa2a6ca17de4f5ab84188ee7f79afbd` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:42.716960 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6565cadc-79c5-40fe-be09-4f4cc56ddff2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=6aa2a6ca17de4f5ab84188ee7f79afbd)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:42.718334 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6565cadc-79c5-40fe-be09-4f4cc56ddff2 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:42.719949 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 589/1172] 10.222.0.23 () {56 vars in 1183 bytes} [Fri May 6 14:46:42 2022] GET /identity/v3/domains/6aa2a6ca17de4f5ab84188ee7f79afbd => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:42.726699 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5fa2d0-f5ec-4710-9898-ceb84b0820ee None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:42.821368 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5fa2d0-f5ec-4710-9898-ceb84b0820ee None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:42.854593 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f22f81e-1861-403e-a951-2487f5f6f05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:42.855236 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f22f81e-1861-403e-a951-2487f5f6f05c None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:42.855513 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f22f81e-1861-403e-a951-2487f5f6f05c None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:42.855782 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f22f81e-1861-403e-a951-2487f5f6f05c None admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:42.856472 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f22f81e-1861-403e-a951-2487f5f6f05c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:42.857904 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f22f81e-1861-403e-a951-2487f5f6f05c None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:42.881739 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 584/1173] 10.222.0.23 () {60 vars in 1141 bytes} [Fri May 6 14:46:42 2022] POST /identity/v3/projects => generated 361 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:46:45.151458 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7079d0-46ec-4278-8d26-83b6ee4db985 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:45.152041 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7079d0-46ec-4278-8d26-83b6ee4db985 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:45.152498 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7079d0-46ec-4278-8d26-83b6ee4db985 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:45.153946 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 590/1174] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:46:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:46:45.159855 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244142b6-1d7b-405e-8a10-c4b0cef7973f None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:45.160300 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244142b6-1d7b-405e-8a10-c4b0cef7973f None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:45.160419 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244142b6-1d7b-405e-8a10-c4b0cef7973f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:45.177024 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-244142b6-1d7b-405e-8a10-c4b0cef7973f None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:46:45.178184 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-244142b6-1d7b-405e-8a10-c4b0cef7973f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:46:45.242128 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 585/1175] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:46:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:46:45.266977 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7079d0-46ec-4278-8d26-83b6ee4db985 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:45.365147 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7079d0-46ec-4278-8d26-83b6ee4db985 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:45.396660 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af9d7f-78f3-4fd7-9b50-e23b9961a2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:45.397499 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af9d7f-78f3-4fd7-9b50-e23b9961a2b2 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:45.397776 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af9d7f-78f3-4fd7-9b50-e23b9961a2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:45.398052 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af9d7f-78f3-4fd7-9b50-e23b9961a2b2 None admin] PATH_INFO: `/v3/domains/6aa2a6ca17de4f5ab84188ee7f79afbd` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:45.399194 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5af9d7f-78f3-4fd7-9b50-e23b9961a2b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=6aa2a6ca17de4f5ab84188ee7f79afbd)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:45.400631 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5af9d7f-78f3-4fd7-9b50-e23b9961a2b2 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:45.402271 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 591/1176] 10.222.0.23 () {56 vars in 1183 bytes} [Fri May 6 14:46:45 2022] GET /identity/v3/domains/6aa2a6ca17de4f5ab84188ee7f79afbd => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:45.408758 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244142b6-1d7b-405e-8a10-c4b0cef7973f None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:45.502191 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244142b6-1d7b-405e-8a10-c4b0cef7973f None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:45.535944 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b7411d-27cc-4447-842c-65aa64fe53c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:45.536593 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7411d-27cc-4447-842c-65aa64fe53c2 None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:45.536901 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7411d-27cc-4447-842c-65aa64fe53c2 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:45.537174 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7411d-27cc-4447-842c-65aa64fe53c2 None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:45.537863 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b7411d-27cc-4447-842c-65aa64fe53c2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:45.539273 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b7411d-27cc-4447-842c-65aa64fe53c2 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:45.559866 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 586/1177] 10.222.0.23 () {60 vars in 1132 bytes} [Fri May 6 14:46:45 2022] POST /identity/v3/users => generated 311 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:46:47.888455 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69706077-5fb0-4935-972c-ef3b02b665e5 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:47.889049 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69706077-5fb0-4935-972c-ef3b02b665e5 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:47.889049 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69706077-5fb0-4935-972c-ef3b02b665e5 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:47.890376 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 592/1178] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:46:47.896529 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f3add-da52-4942-b3e9-5f996f4f32fe None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:47.896958 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f3add-da52-4942-b3e9-5f996f4f32fe None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:47.897129 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f3add-da52-4942-b3e9-5f996f4f32fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:47.913783 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-7e4f3add-da52-4942-b3e9-5f996f4f32fe None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:46:47.914959 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-7e4f3add-da52-4942-b3e9-5f996f4f32fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:46:47.978532 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 587/1179] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:46:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:46:48.003313 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69706077-5fb0-4935-972c-ef3b02b665e5 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:48.100832 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69706077-5fb0-4935-972c-ef3b02b665e5 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:48.134595 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d940fa-2642-4f3e-9eea-fa7c48e9fa87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:48.135417 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d940fa-2642-4f3e-9eea-fa7c48e9fa87 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:48.135741 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d940fa-2642-4f3e-9eea-fa7c48e9fa87 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:48.136051 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d940fa-2642-4f3e-9eea-fa7c48e9fa87 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:48.145927 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d940fa-2642-4f3e-9eea-fa7c48e9fa87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:46:48.153505 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-97d940fa-2642-4f3e-9eea-fa7c48e9fa87 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:46:48.153505 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:46:48.153505 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:46:48.153505 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:46:48.153505 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:46:48.153505 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:46:48.153505 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:46:48.153505 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:46:48.153505 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:46:48.153505 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:46:48.153505 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:46:48.153505 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:46:48.153505 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:46:48.153505 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:46:48.153505 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:46:48.153505 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:46:48.154492 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:46:48.154492 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:46:48.154492 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:46:48.154492 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:46:48.154492 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:46:48.154492 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:46:48.154492 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:46:48.154492 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:46:48.154492 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:46:48.154492 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:46:48.154492 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:46:48.154492 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:46:48.154492 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:46:48.154492 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:46:48.154492 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:46:48.154492 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:46:48.155610 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:46:48.155610 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:46:48.155610 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:46:48.155610 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:46:48.155610 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:46:48.155610 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:46:48.155610 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:46:48.155610 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:46:48.155610 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:46:48.155610 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:46:48.155610 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:46:48.155610 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:46:48.156916 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 593/1180] 10.222.0.23 () {56 vars in 1096 bytes} [Fri May 6 14:46:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:46:48.161115 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4f3add-da52-4942-b3e9-5f996f4f32fe None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:48.255543 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4f3add-da52-4942-b3e9-5f996f4f32fe None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:48.287238 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cbbd59-af47-4d5d-bd07-4d4745abeea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:48.287892 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbbd59-af47-4d5d-bd07-4d4745abeea9 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:48.288174 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbbd59-af47-4d5d-bd07-4d4745abeea9 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:48.288450 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbbd59-af47-4d5d-bd07-4d4745abeea9 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:48.289207 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cbbd59-af47-4d5d-bd07-4d4745abeea9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:46:48.289513 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cbbd59-af47-4d5d-bd07-4d4745abeea9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:48.290876 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cbbd59-af47-4d5d-bd07-4d4745abeea9 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:48.299111 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 588/1181] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:46:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:48.306621 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d940fa-2642-4f3e-9eea-fa7c48e9fa87 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:48.405194 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d940fa-2642-4f3e-9eea-fa7c48e9fa87 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:48.436551 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff74b1b-f80d-4f3c-a967-ad87855aff99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:48.437428 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff74b1b-f80d-4f3c-a967-ad87855aff99 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:48.437706 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff74b1b-f80d-4f3c-a967-ad87855aff99 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:48.438062 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff74b1b-f80d-4f3c-a967-ad87855aff99 None admin] PATH_INFO: `/v3/users/14a59add6a8f4c4592102622f82cf0d4` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:48.451715 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff74b1b-f80d-4f3c-a967-ad87855aff99 None admin] RBAC: Authorizing `identity:get_user(user_id=14a59add6a8f4c4592102622f82cf0d4)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:48.453196 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff74b1b-f80d-4f3c-a967-ad87855aff99 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:48.455425 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 594/1182] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:46:48 2022] GET /identity/v3/users/14a59add6a8f4c4592102622f82cf0d4 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:48.461352 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cbbd59-af47-4d5d-bd07-4d4745abeea9 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:48.557108 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cbbd59-af47-4d5d-bd07-4d4745abeea9 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:48.589359 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62b820c-5a20-465d-acec-a75ac77d585a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:48.589998 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b820c-5a20-465d-acec-a75ac77d585a None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:48.590282 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b820c-5a20-465d-acec-a75ac77d585a None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:48.590561 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b820c-5a20-465d-acec-a75ac77d585a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:48.653518 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62b820c-5a20-465d-acec-a75ac77d585a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:48.655278 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62b820c-5a20-465d-acec-a75ac77d585a None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:48.747585 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 589/1183] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:46:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 06 14:46:48.754378 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff74b1b-f80d-4f3c-a967-ad87855aff99 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:48.848541 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff74b1b-f80d-4f3c-a967-ad87855aff99 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:48.881951 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e3f14-f10d-4dfa-8e81-183cd3f44e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:48.882784 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e3f14-f10d-4dfa-8e81-183cd3f44e3a None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:48.883058 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e3f14-f10d-4dfa-8e81-183cd3f44e3a None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:48.883324 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e3f14-f10d-4dfa-8e81-183cd3f44e3a None admin] PATH_INFO: `/v3/projects/9cb8198fe3944a3fa45ac40da1b45daa` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:48.884459 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010e3f14-f10d-4dfa-8e81-183cd3f44e3a None admin] RBAC: Authorizing `identity:get_project(project_id=9cb8198fe3944a3fa45ac40da1b45daa)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:48.885891 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010e3f14-f10d-4dfa-8e81-183cd3f44e3a None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:48.887546 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 595/1184] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:46:48 2022] GET /identity/v3/projects/9cb8198fe3944a3fa45ac40da1b45daa => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:48.893985 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62b820c-5a20-465d-acec-a75ac77d585a None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:48.988208 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62b820c-5a20-465d-acec-a75ac77d585a None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:49.020357 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3664e-a8e3-4bd5-bd5d-5653107224d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:49.021027 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3664e-a8e3-4bd5-bd5d-5653107224d6 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:49.021312 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3664e-a8e3-4bd5-bd5d-5653107224d6 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:49.021590 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3664e-a8e3-4bd5-bd5d-5653107224d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:49.022400 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e3664e-a8e3-4bd5-bd5d-5653107224d6 None admin] RBAC: Adding query filter params (role.id=6dcd35d1a83748759d6ef9f026077835, scope.project.id=9cb8198fe3944a3fa45ac40da1b45daa, user.id=14a59add6a8f4c4592102622f82cf0d4) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:46:49.022712 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e3664e-a8e3-4bd5-bd5d-5653107224d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:49.024126 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e3664e-a8e3-4bd5-bd5d-5653107224d6 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:49.029344 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 590/1185] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:46:48 2022] GET /identity/v3/role_assignments?user.id=14a59add6a8f4c4592102622f82cf0d4&scope.project.id=9cb8198fe3944a3fa45ac40da1b45daa&role.id=6dcd35d1a83748759d6ef9f026077835 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:51.244862 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b984f61-bfac-48d8-bb33-923ca84cb0ac None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:51.245368 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b984f61-bfac-48d8-bb33-923ca84cb0ac None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:51.245474 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b984f61-bfac-48d8-bb33-923ca84cb0ac None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:51.246772 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 596/1186] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:46:51.252756 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b8346-9c4f-4df0-8948-4220431d8cf5 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:51.253577 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b8346-9c4f-4df0-8948-4220431d8cf5 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:51.253929 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b8346-9c4f-4df0-8948-4220431d8cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:51.270397 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-045b8346-9c4f-4df0-8948-4220431d8cf5 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:46:51.271801 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-045b8346-9c4f-4df0-8948-4220431d8cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:46:51.341588 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 591/1187] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:46:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:46:51.365483 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b984f61-bfac-48d8-bb33-923ca84cb0ac None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:51.464884 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b984f61-bfac-48d8-bb33-923ca84cb0ac None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:51.498438 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f3aaa4-73bf-43d3-b41b-0247ae7ce2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:51.499232 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f3aaa4-73bf-43d3-b41b-0247ae7ce2a9 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:51.499510 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f3aaa4-73bf-43d3-b41b-0247ae7ce2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:51.499781 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f3aaa4-73bf-43d3-b41b-0247ae7ce2a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:51.509549 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f3aaa4-73bf-43d3-b41b-0247ae7ce2a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:46:51.516952 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-38f3aaa4-73bf-43d3-b41b-0247ae7ce2a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:46:51.516952 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:46:51.516952 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:46:51.516952 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:46:51.516952 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:46:51.516952 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:46:51.516952 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:46:51.516952 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:46:51.516952 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:46:51.516952 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:46:51.516952 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:46:51.516952 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:46:51.516952 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:46:51.516952 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:46:51.516952 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:46:51.516952 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:46:51.518032 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:46:51.518032 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:46:51.518032 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:46:51.518032 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:46:51.518032 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:46:51.518032 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:46:51.518032 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:46:51.518032 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:46:51.518032 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:46:51.518032 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:46:51.518032 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:46:51.518032 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:46:51.518032 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:46:51.518032 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:46:51.518032 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:46:51.518032 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:46:51.519091 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:46:51.519091 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:46:51.519091 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:46:51.519091 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:46:51.519091 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:46:51.519091 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:46:51.519091 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:46:51.519091 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:46:51.519091 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:46:51.519091 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:46:51.519091 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:46:51.519091 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:46:51.519091 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 597/1188] 10.222.0.23 () {56 vars in 1096 bytes} [Fri May 6 14:46:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:46:51.524901 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045b8346-9c4f-4df0-8948-4220431d8cf5 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:51.620052 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045b8346-9c4f-4df0-8948-4220431d8cf5 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:51.654342 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f32c91-ed5e-4e42-868c-cf567580f56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:51.655252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f32c91-ed5e-4e42-868c-cf567580f56f None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:51.655531 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f32c91-ed5e-4e42-868c-cf567580f56f None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:51.655802 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f32c91-ed5e-4e42-868c-cf567580f56f None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:51.656524 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f32c91-ed5e-4e42-868c-cf567580f56f None admin] RBAC: Adding query filter params (name=admin) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:46:51.656852 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f32c91-ed5e-4e42-868c-cf567580f56f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:51.659043 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f32c91-ed5e-4e42-868c-cf567580f56f None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:51.666642 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 592/1189] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:46:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:51.672640 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f3aaa4-73bf-43d3-b41b-0247ae7ce2a9 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:51.767589 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f3aaa4-73bf-43d3-b41b-0247ae7ce2a9 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:51.799890 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee39d9-dbcf-4c14-8228-131605b789da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:51.800828 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee39d9-dbcf-4c14-8228-131605b789da None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:51.801145 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee39d9-dbcf-4c14-8228-131605b789da None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:51.801425 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee39d9-dbcf-4c14-8228-131605b789da None admin] PATH_INFO: `/v3/users/14a59add6a8f4c4592102622f82cf0d4` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:51.802624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ee39d9-dbcf-4c14-8228-131605b789da None admin] RBAC: Authorizing `identity:get_user(user_id=14a59add6a8f4c4592102622f82cf0d4)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:51.804046 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ee39d9-dbcf-4c14-8228-131605b789da None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:51.805683 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 598/1190] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:46:51 2022] GET /identity/v3/users/14a59add6a8f4c4592102622f82cf0d4 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:51.811388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f32c91-ed5e-4e42-868c-cf567580f56f None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:51.907467 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f32c91-ed5e-4e42-868c-cf567580f56f None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:51.939300 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974263b0-bba0-49ea-9631-a4825393147f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:51.940304 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974263b0-bba0-49ea-9631-a4825393147f None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:51.940582 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974263b0-bba0-49ea-9631-a4825393147f None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:51.940882 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974263b0-bba0-49ea-9631-a4825393147f None admin] PATH_INFO: `/v3/projects/9cb8198fe3944a3fa45ac40da1b45daa` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:51.942029 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974263b0-bba0-49ea-9631-a4825393147f None admin] RBAC: Authorizing `identity:get_project(project_id=9cb8198fe3944a3fa45ac40da1b45daa)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:51.943436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974263b0-bba0-49ea-9631-a4825393147f None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:51.945119 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 593/1191] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:46:51 2022] GET /identity/v3/projects/9cb8198fe3944a3fa45ac40da1b45daa => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:51.951090 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee39d9-dbcf-4c14-8228-131605b789da None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:52.045412 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee39d9-dbcf-4c14-8228-131605b789da None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:52.076539 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a5306-5834-4105-a2f7-0b662c6f4359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:52.077444 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a5306-5834-4105-a2f7-0b662c6f4359 None admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:52.077722 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a5306-5834-4105-a2f7-0b662c6f4359 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:52.077991 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a5306-5834-4105-a2f7-0b662c6f4359 None admin] PATH_INFO: `/v3/projects/9cb8198fe3944a3fa45ac40da1b45daa/users/14a59add6a8f4c4592102622f82cf0d4/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:52.080541 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5a5306-5834-4105-a2f7-0b662c6f4359 None admin] RBAC: Authorizing `identity:create_grant(project_id=9cb8198fe3944a3fa45ac40da1b45daa, user_id=14a59add6a8f4c4592102622f82cf0d4, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:52.081996 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5a5306-5834-4105-a2f7-0b662c6f4359 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:52.094212 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 599/1192] 10.222.0.23 () {58 vars in 1439 bytes} [Fri May 6 14:46:51 2022] PUT /identity/v3/projects/9cb8198fe3944a3fa45ac40da1b45daa/users/14a59add6a8f4c4592102622f82cf0d4/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:46:54.309615 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749c704-af71-4001-8185-dc8b8ae8cd63 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:54.310173 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749c704-af71-4001-8185-dc8b8ae8cd63 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:54.310173 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749c704-af71-4001-8185-dc8b8ae8cd63 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:54.311518 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 594/1193] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:46:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:46:54.317819 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c7217-c6d0-49f7-baee-8088fb1eb707 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:54.318252 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c7217-c6d0-49f7-baee-8088fb1eb707 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:54.318381 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c7217-c6d0-49f7-baee-8088fb1eb707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:54.334779 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-be4c7217-c6d0-49f7-baee-8088fb1eb707 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:46:54.335981 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-be4c7217-c6d0-49f7-baee-8088fb1eb707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:46:54.399903 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 600/1194] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:46:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:46:54.423915 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1749c704-af71-4001-8185-dc8b8ae8cd63 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:54.527468 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1749c704-af71-4001-8185-dc8b8ae8cd63 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:54.561042 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a03207-b777-4351-8c84-926620069bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:54.561823 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a03207-b777-4351-8c84-926620069bc1 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:54.562103 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a03207-b777-4351-8c84-926620069bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:54.562376 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a03207-b777-4351-8c84-926620069bc1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:54.572967 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a03207-b777-4351-8c84-926620069bc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:46:54.581019 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-e0a03207-b777-4351-8c84-926620069bc1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:46:54.581019 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:46:54.581019 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:46:54.581019 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:46:54.581019 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:46:54.581019 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:46:54.581019 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:46:54.581019 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:46:54.581019 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:46:54.581019 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:46:54.581019 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:46:54.581019 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:46:54.581019 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:46:54.581019 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:46:54.581019 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:46:54.581019 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:46:54.582120 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:46:54.582120 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:46:54.582120 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:46:54.582120 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:46:54.582120 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:46:54.582120 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:46:54.582120 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:46:54.582120 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:46:54.582120 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:46:54.582120 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:46:54.582120 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:46:54.582120 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:46:54.582120 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:46:54.582120 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:46:54.582120 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:46:54.582120 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:46:54.583176 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:46:54.583176 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:46:54.583176 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:46:54.583176 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:46:54.583176 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:46:54.583176 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:46:54.583176 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:46:54.583176 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:46:54.583176 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:46:54.583176 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:46:54.583176 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:46:54.583176 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:46:54.583176 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 595/1195] 10.222.0.23 () {56 vars in 1096 bytes} [Fri May 6 14:46:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:46:54.588772 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4c7217-c6d0-49f7-baee-8088fb1eb707 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:54.681082 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4c7217-c6d0-49f7-baee-8088fb1eb707 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:54.713377 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e331eb7-33c7-47cc-bc0a-35cc3ebe41fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:54.714025 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e331eb7-33c7-47cc-bc0a-35cc3ebe41fd None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:54.714304 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e331eb7-33c7-47cc-bc0a-35cc3ebe41fd None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:54.714575 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e331eb7-33c7-47cc-bc0a-35cc3ebe41fd None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:54.715297 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e331eb7-33c7-47cc-bc0a-35cc3ebe41fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:46:54.715598 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e331eb7-33c7-47cc-bc0a-35cc3ebe41fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:54.716979 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e331eb7-33c7-47cc-bc0a-35cc3ebe41fd None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:54.724530 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 601/1196] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:46:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:54.730218 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a03207-b777-4351-8c84-926620069bc1 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:54.828446 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a03207-b777-4351-8c84-926620069bc1 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:54.860158 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e21d4fe-72a3-4061-9d99-9faf9a0a3515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:54.862129 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e21d4fe-72a3-4061-9d99-9faf9a0a3515 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:54.862418 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e21d4fe-72a3-4061-9d99-9faf9a0a3515 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:54.862688 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e21d4fe-72a3-4061-9d99-9faf9a0a3515 None admin] PATH_INFO: `/v3/users/14a59add6a8f4c4592102622f82cf0d4` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:54.863847 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e21d4fe-72a3-4061-9d99-9faf9a0a3515 None admin] RBAC: Authorizing `identity:get_user(user_id=14a59add6a8f4c4592102622f82cf0d4)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:54.865287 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e21d4fe-72a3-4061-9d99-9faf9a0a3515 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:54.866855 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 596/1197] 10.222.0.23 () {56 vars in 1177 bytes} [Fri May 6 14:46:54 2022] GET /identity/v3/users/14a59add6a8f4c4592102622f82cf0d4 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:54.872577 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e331eb7-33c7-47cc-bc0a-35cc3ebe41fd None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:54.968521 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e331eb7-33c7-47cc-bc0a-35cc3ebe41fd None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:55.000447 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e519dc-ce46-4673-b21a-8df3bbd430d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:55.001117 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e519dc-ce46-4673-b21a-8df3bbd430d8 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:55.001440 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e519dc-ce46-4673-b21a-8df3bbd430d8 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:55.002241 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e519dc-ce46-4673-b21a-8df3bbd430d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:55.065469 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e519dc-ce46-4673-b21a-8df3bbd430d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:55.066901 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e519dc-ce46-4673-b21a-8df3bbd430d8 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:55.161326 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 602/1198] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:46:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 06 14:46:55.167099 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e21d4fe-72a3-4061-9d99-9faf9a0a3515 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:55.261388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e21d4fe-72a3-4061-9d99-9faf9a0a3515 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:55.293524 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37248556-ecaa-4c62-8346-ac8dd51a7f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:55.294368 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37248556-ecaa-4c62-8346-ac8dd51a7f65 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:55.294651 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37248556-ecaa-4c62-8346-ac8dd51a7f65 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:55.294927 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37248556-ecaa-4c62-8346-ac8dd51a7f65 None admin] PATH_INFO: `/v3/projects/9cb8198fe3944a3fa45ac40da1b45daa` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:55.296088 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37248556-ecaa-4c62-8346-ac8dd51a7f65 None admin] RBAC: Authorizing `identity:get_project(project_id=9cb8198fe3944a3fa45ac40da1b45daa)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:55.297535 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37248556-ecaa-4c62-8346-ac8dd51a7f65 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:55.299834 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 597/1199] 10.222.0.23 () {56 vars in 1186 bytes} [Fri May 6 14:46:55 2022] GET /identity/v3/projects/9cb8198fe3944a3fa45ac40da1b45daa => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:46:55.305889 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e519dc-ce46-4673-b21a-8df3bbd430d8 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:46:55.398614 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e519dc-ce46-4673-b21a-8df3bbd430d8 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:46:55.431178 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf66109-08e7-4088-968b-c39938d685a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:46:55.431817 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf66109-08e7-4088-968b-c39938d685a6 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:46:55.432097 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf66109-08e7-4088-968b-c39938d685a6 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:46:55.432372 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf66109-08e7-4088-968b-c39938d685a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:46:55.433214 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf66109-08e7-4088-968b-c39938d685a6 None admin] RBAC: Adding query filter params (role.id=6dcd35d1a83748759d6ef9f026077835, scope.project.id=9cb8198fe3944a3fa45ac40da1b45daa, user.id=14a59add6a8f4c4592102622f82cf0d4) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:46:55.433523 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf66109-08e7-4088-968b-c39938d685a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:46:55.434945 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf66109-08e7-4088-968b-c39938d685a6 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:46:55.440999 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 603/1200] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:46:55 2022] GET /identity/v3/role_assignments?user.id=14a59add6a8f4c4592102622f82cf0d4&scope.project.id=9cb8198fe3944a3fa45ac40da1b45daa&role.id=6dcd35d1a83748759d6ef9f026077835 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:47:56.604626 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d7b86-9bfd-41c1-9230-e424b0f1cbfc None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:47:56.605238 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d7b86-9bfd-41c1-9230-e424b0f1cbfc None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:47:56.605330 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d7b86-9bfd-41c1-9230-e424b0f1cbfc None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:47:56.607048 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 598/1201] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:47:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:47:56.613633 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dfc08e-6755-42d0-8b04-c27bb7d6dc92 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:47:56.614143 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dfc08e-6755-42d0-8b04-c27bb7d6dc92 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:47:56.614230 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dfc08e-6755-42d0-8b04-c27bb7d6dc92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:47:56.641705 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-c7dfc08e-6755-42d0-8b04-c27bb7d6dc92 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:47:56.642936 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-c7dfc08e-6755-42d0-8b04-c27bb7d6dc92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:47:56.728497 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 604/1202] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:47:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:47:56.753696 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93d7b86-9bfd-41c1-9230-e424b0f1cbfc None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:47:56.881272 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93d7b86-9bfd-41c1-9230-e424b0f1cbfc None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:47:56.923248 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ab9a1f-084f-4c0b-9c3d-0769d91aa35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:47:56.924057 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ab9a1f-084f-4c0b-9c3d-0769d91aa35e None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:47:56.924403 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ab9a1f-084f-4c0b-9c3d-0769d91aa35e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:47:56.924631 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ab9a1f-084f-4c0b-9c3d-0769d91aa35e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:47:56.936876 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ab9a1f-084f-4c0b-9c3d-0769d91aa35e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:47:56.938724 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ab9a1f-084f-4c0b-9c3d-0769d91aa35e None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:47:56.958603 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-35ab9a1f-084f-4c0b-9c3d-0769d91aa35e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:47:56.958603 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:47:56.958603 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:47:56.958603 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:47:56.958603 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:47:56.958603 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:47:56.958603 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:47:56.958603 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:47:56.958603 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:47:56.958603 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:47:56.958603 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:47:56.958603 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:47:56.958603 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:47:56.958603 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:47:56.958603 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:47:56.960106 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:47:56.960106 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:47:56.960106 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:47:56.960106 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:47:56.960106 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:47:56.960106 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:47:56.960106 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:47:56.960106 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:47:56.960106 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:47:56.960106 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:47:56.960106 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:47:56.960106 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:47:56.960106 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:47:56.960106 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:47:56.960106 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:47:56.960106 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:47:56.960106 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:47:56.961370 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:47:56.961370 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:47:56.961370 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:47:56.961370 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:47:56.961370 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 599/1203] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:47:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:47:56.967632 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dfc08e-6755-42d0-8b04-c27bb7d6dc92 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:47:57.090287 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dfc08e-6755-42d0-8b04-c27bb7d6dc92 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:47:57.136245 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07565058-d1a1-4fe4-84c4-b44a007df256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:47:57.137042 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07565058-d1a1-4fe4-84c4-b44a007df256 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:47:57.137403 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07565058-d1a1-4fe4-84c4-b44a007df256 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:47:57.137779 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07565058-d1a1-4fe4-84c4-b44a007df256 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:47:57.138653 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07565058-d1a1-4fe4-84c4-b44a007df256 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:47:57.139205 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07565058-d1a1-4fe4-84c4-b44a007df256 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:47:57.140886 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07565058-d1a1-4fe4-84c4-b44a007df256 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:47:57.156000 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 605/1204] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:47:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:47:57.159478 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ab9a1f-084f-4c0b-9c3d-0769d91aa35e None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:47:57.275856 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ab9a1f-084f-4c0b-9c3d-0769d91aa35e None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:47:57.324613 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad3a4c6-7a74-4a66-b66c-8f403978b586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:47:57.325423 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3a4c6-7a74-4a66-b66c-8f403978b586 None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:47:57.325708 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3a4c6-7a74-4a66-b66c-8f403978b586 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:47:57.325986 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3a4c6-7a74-4a66-b66c-8f403978b586 None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:47:57.327249 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad3a4c6-7a74-4a66-b66c-8f403978b586 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:47:57.328932 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad3a4c6-7a74-4a66-b66c-8f403978b586 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:47:57.356568 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 600/1205] 10.222.0.23 () {60 vars in 1132 bytes} [Fri May 6 14:47:57 2022] POST /identity/v3/users => generated 251 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:48:00.060346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca5eb6-2f9d-4321-846c-2ef0688379f1 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:00.061012 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca5eb6-2f9d-4321-846c-2ef0688379f1 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:00.061012 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca5eb6-2f9d-4321-846c-2ef0688379f1 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:00.062353 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 606/1206] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:48:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:48:00.072590 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d138bba-4a43-44c1-8d16-df4829221607 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:00.072590 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d138bba-4a43-44c1-8d16-df4829221607 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:00.072590 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d138bba-4a43-44c1-8d16-df4829221607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:00.091204 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-2d138bba-4a43-44c1-8d16-df4829221607 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:48:00.092720 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-2d138bba-4a43-44c1-8d16-df4829221607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:48:00.167621 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 601/1207] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:48:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:48:00.193126 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca5eb6-2f9d-4321-846c-2ef0688379f1 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:00.311720 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca5eb6-2f9d-4321-846c-2ef0688379f1 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:00.355682 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e960f890-cfd5-4a48-9595-6d8f1b579dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:00.356826 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960f890-cfd5-4a48-9595-6d8f1b579dbb None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:00.357026 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960f890-cfd5-4a48-9595-6d8f1b579dbb None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:00.357413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960f890-cfd5-4a48-9595-6d8f1b579dbb None admin] PATH_INFO: `/v3/roles/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:00.370704 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e960f890-cfd5-4a48-9595-6d8f1b579dbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:48:00.379074 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-e960f890-cfd5-4a48-9595-6d8f1b579dbb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 06 14:48:00.379074 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:48:00.379074 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:48:00.379074 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:48:00.379074 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:48:00.379074 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:48:00.379074 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:48:00.379074 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:48:00.379074 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:48:00.379074 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:48:00.379074 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:48:00.379074 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:48:00.379074 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:48:00.379074 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:48:00.379074 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:48:00.379074 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:48:00.380188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:48:00.380188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:48:00.380188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:48:00.380188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:48:00.380188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:48:00.380188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:48:00.380188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:48:00.380188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:48:00.380188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:48:00.380188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:48:00.380188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:48:00.380188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:48:00.380188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:48:00.380188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:48:00.380188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:48:00.380188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:48:00.381083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:48:00.381083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:48:00.381083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:48:00.381083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:48:00.381083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:48:00.381083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:48:00.381083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:48:00.381083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:48:00.381083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:48:00.381083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:48:00.381083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 06 14:48:00.381083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:48:00.381083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 607/1208] 10.222.0.23 () {56 vars in 1102 bytes} [Fri May 6 14:48:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:48:00.390865 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d138bba-4a43-44c1-8d16-df4829221607 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:00.502584 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d138bba-4a43-44c1-8d16-df4829221607 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:00.538678 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf7854d-d897-4d3f-b4dc-b353ca42a677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:00.539465 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7854d-d897-4d3f-b4dc-b353ca42a677 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:00.539755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7854d-d897-4d3f-b4dc-b353ca42a677 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:00.540063 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7854d-d897-4d3f-b4dc-b353ca42a677 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:00.540859 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf7854d-d897-4d3f-b4dc-b353ca42a677 None admin] RBAC: Adding query filter params (name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:48:00.541216 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf7854d-d897-4d3f-b4dc-b353ca42a677 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:00.542609 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf7854d-d897-4d3f-b4dc-b353ca42a677 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:00.553435 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 602/1209] 10.222.0.23 () {56 vars in 1103 bytes} [Fri May 6 14:48:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:48:00.559476 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e960f890-cfd5-4a48-9595-6d8f1b579dbb None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:00.676411 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e960f890-cfd5-4a48-9595-6d8f1b579dbb None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:00.713274 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa519da9-ef4b-42bc-b6d3-f5f1ef1ed086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:00.714152 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa519da9-ef4b-42bc-b6d3-f5f1ef1ed086 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:00.714419 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa519da9-ef4b-42bc-b6d3-f5f1ef1ed086 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:00.714730 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa519da9-ef4b-42bc-b6d3-f5f1ef1ed086 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:00.727746 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa519da9-ef4b-42bc-b6d3-f5f1ef1ed086 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:00.728591 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa519da9-ef4b-42bc-b6d3-f5f1ef1ed086 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:00.747258 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-aa519da9-ef4b-42bc-b6d3-f5f1ef1ed086 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:48:00.747258 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:48:00.747258 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:48:00.747258 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:48:00.747258 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:48:00.747258 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:48:00.747258 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:48:00.747258 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:48:00.747258 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:48:00.747258 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:48:00.747258 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:48:00.747258 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:48:00.747258 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:48:00.747258 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:48:00.747258 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:48:00.748586 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:48:00.748586 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:48:00.748586 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:48:00.748586 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:48:00.748586 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:48:00.748586 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:48:00.748586 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:48:00.748586 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:48:00.748586 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:48:00.748586 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:48:00.748586 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:48:00.748586 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:48:00.748586 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:48:00.748586 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:48:00.748586 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:48:00.748586 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:48:00.748586 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:48:00.749913 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:48:00.749913 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:48:00.749913 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:48:00.749913 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:48:00.749913 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 608/1210] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:48:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:48:00.754573 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf7854d-d897-4d3f-b4dc-b353ca42a677 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:00.876022 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf7854d-d897-4d3f-b4dc-b353ca42a677 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:00.921363 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288553ed-76b4-4a77-9c91-181c92937ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:00.922137 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288553ed-76b4-4a77-9c91-181c92937ceb None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:00.922459 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288553ed-76b4-4a77-9c91-181c92937ceb None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:00.922794 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288553ed-76b4-4a77-9c91-181c92937ceb None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:00.923552 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288553ed-76b4-4a77-9c91-181c92937ceb None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:48:00.923857 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288553ed-76b4-4a77-9c91-181c92937ceb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:00.925256 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288553ed-76b4-4a77-9c91-181c92937ceb None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:00.939288 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 603/1211] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:48:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:48:00.945010 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa519da9-ef4b-42bc-b6d3-f5f1ef1ed086 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:01.069288 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa519da9-ef4b-42bc-b6d3-f5f1ef1ed086 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:01.107447 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87782cf-a432-446e-aa68-7f31e05b0296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:01.108715 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87782cf-a432-446e-aa68-7f31e05b0296 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:01.109121 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87782cf-a432-446e-aa68-7f31e05b0296 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:01.109394 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87782cf-a432-446e-aa68-7f31e05b0296 None admin] PATH_INFO: `/v3/users/placement` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:01.118257 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87782cf-a432-446e-aa68-7f31e05b0296 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:01.119867 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87782cf-a432-446e-aa68-7f31e05b0296 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:01.135329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-c87782cf-a432-446e-aa68-7f31e05b0296 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 06 14:48:01.135329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:48:01.135329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:48:01.135329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:48:01.135329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:48:01.135329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:48:01.135329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:48:01.135329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:48:01.135329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:48:01.135329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:48:01.135329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:48:01.135329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:48:01.135329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 06 14:48:01.135329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_user(user_id) May 06 14:48:01.135329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 06 14:48:01.135329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 06 14:48:01.136864 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:48:01.136864 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:48:01.136864 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 06 14:48:01.136864 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:48:01.136864 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 06 14:48:01.136864 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:48:01.136864 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:48:01.136864 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:48:01.136864 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:48:01.136864 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:48:01.136864 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:48:01.136864 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:48:01.136864 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:48:01.136864 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:48:01.136864 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:48:01.136864 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:48:01.136864 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:48:01.136864 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:48:01.137953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:48:01.137953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:48:01.137953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 06 14:48:01.137953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 06 14:48:01.137953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 06 14:48:01.137953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 06 14:48:01.137953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 06 14:48:01.137953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 06 14:48:01.137953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 06 14:48:01.137953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:48:01.137953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 609/1212] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:48:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:48:01.143045 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288553ed-76b4-4a77-9c91-181c92937ceb None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:01.262439 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288553ed-76b4-4a77-9c91-181c92937ceb None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:01.303515 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12486f57-cadc-4c29-8820-00ea14b34bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:01.304349 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12486f57-cadc-4c29-8820-00ea14b34bcc None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:01.304741 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12486f57-cadc-4c29-8820-00ea14b34bcc None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:01.305080 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12486f57-cadc-4c29-8820-00ea14b34bcc None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:01.306191 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12486f57-cadc-4c29-8820-00ea14b34bcc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:48:01.306733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12486f57-cadc-4c29-8820-00ea14b34bcc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:01.308428 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12486f57-cadc-4c29-8820-00ea14b34bcc None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:01.329453 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 604/1213] 10.222.0.23 () {56 vars in 1143 bytes} [Fri May 6 14:48:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:48:01.335051 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87782cf-a432-446e-aa68-7f31e05b0296 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:01.455100 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87782cf-a432-446e-aa68-7f31e05b0296 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:01.495391 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e5d1d3-6e2e-4c64-a9af-66457f270e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:01.496070 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e5d1d3-6e2e-4c64-a9af-66457f270e85 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:01.496357 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e5d1d3-6e2e-4c64-a9af-66457f270e85 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:01.496660 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e5d1d3-6e2e-4c64-a9af-66457f270e85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:01.577691 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e5d1d3-6e2e-4c64-a9af-66457f270e85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:01.579105 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e5d1d3-6e2e-4c64-a9af-66457f270e85 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:01.688924 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 610/1214] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:48:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 06 14:48:01.694624 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12486f57-cadc-4c29-8820-00ea14b34bcc None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:01.804444 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12486f57-cadc-4c29-8820-00ea14b34bcc None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:01.843535 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac89d9-0e5f-4967-a1b4-9376aeb9b92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:01.844409 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac89d9-0e5f-4967-a1b4-9376aeb9b92d None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:01.844794 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac89d9-0e5f-4967-a1b4-9376aeb9b92d None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:01.845169 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac89d9-0e5f-4967-a1b4-9376aeb9b92d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:01.856258 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ac89d9-0e5f-4967-a1b4-9376aeb9b92d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:01.858756 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ac89d9-0e5f-4967-a1b4-9376aeb9b92d None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:01.877330 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-c8ac89d9-0e5f-4967-a1b4-9376aeb9b92d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:48:01.877330 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:48:01.877330 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:48:01.877330 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:48:01.877330 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:48:01.877330 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:48:01.877330 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:48:01.877330 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:48:01.877330 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:48:01.877330 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:48:01.877330 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:48:01.877330 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:48:01.877330 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:48:01.877330 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:48:01.877330 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:48:01.878846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:48:01.878846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:48:01.878846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:48:01.878846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:48:01.878846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:48:01.878846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:48:01.878846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:48:01.878846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:48:01.878846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:48:01.878846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:48:01.878846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:48:01.878846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:48:01.878846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:48:01.878846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:48:01.878846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:48:01.878846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:48:01.878846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:48:01.879783 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:48:01.879783 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:48:01.879783 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:48:01.879783 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:48:01.879783 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 605/1215] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:48:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:48:01.884384 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e5d1d3-6e2e-4c64-a9af-66457f270e85 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:02.002899 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e5d1d3-6e2e-4c64-a9af-66457f270e85 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:02.039084 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4253aeea-f733-4f65-94f2-f4501a7badd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:02.039754 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253aeea-f733-4f65-94f2-f4501a7badd9 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:02.040040 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253aeea-f733-4f65-94f2-f4501a7badd9 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:02.040317 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253aeea-f733-4f65-94f2-f4501a7badd9 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:02.041075 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4253aeea-f733-4f65-94f2-f4501a7badd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:48:02.041382 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4253aeea-f733-4f65-94f2-f4501a7badd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:02.042761 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4253aeea-f733-4f65-94f2-f4501a7badd9 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:02.054646 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 611/1216] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:48:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:48:02.061084 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac89d9-0e5f-4967-a1b4-9376aeb9b92d None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:02.181803 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac89d9-0e5f-4967-a1b4-9376aeb9b92d None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:02.222390 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62bbcf7-39fa-4eb7-b05a-9ee497a1bc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:02.223178 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62bbcf7-39fa-4eb7-b05a-9ee497a1bc3c None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:02.223496 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62bbcf7-39fa-4eb7-b05a-9ee497a1bc3c None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:02.223761 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62bbcf7-39fa-4eb7-b05a-9ee497a1bc3c None admin] PATH_INFO: `/v3/projects/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:02.230813 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62bbcf7-39fa-4eb7-b05a-9ee497a1bc3c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:02.233084 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62bbcf7-39fa-4eb7-b05a-9ee497a1bc3c None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:02.248815 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-b62bbcf7-39fa-4eb7-b05a-9ee497a1bc3c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:48:02.248815 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:48:02.248815 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:48:02.248815 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:48:02.248815 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:48:02.248815 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:48:02.248815 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:48:02.248815 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:48:02.248815 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:48:02.248815 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:48:02.248815 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:48:02.248815 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:48:02.248815 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 06 14:48:02.248815 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(project_id) May 06 14:48:02.248815 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 06 14:48:02.249862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 06 14:48:02.249862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:48:02.249862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:48:02.249862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:48:02.249862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:48:02.249862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:48:02.249862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:48:02.249862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:48:02.249862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:48:02.249862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:48:02.249862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:48:02.249862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:48:02.249862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:48:02.249862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:48:02.249862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:48:02.249862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:48:02.249862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:48:02.250854 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 06 14:48:02.250854 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 06 14:48:02.250854 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 06 14:48:02.250854 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 06 14:48:02.250854 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 06 14:48:02.250854 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 06 14:48:02.250854 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:48:02.250854 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:48:02.250854 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 606/1217] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:48:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:48:02.256234 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4253aeea-f733-4f65-94f2-f4501a7badd9 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:02.376516 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4253aeea-f733-4f65-94f2-f4501a7badd9 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:02.425129 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf25f5-6ac9-42b6-b1fe-eab26d379809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:02.425719 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf25f5-6ac9-42b6-b1fe-eab26d379809 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:02.426006 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf25f5-6ac9-42b6-b1fe-eab26d379809 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:02.426279 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf25f5-6ac9-42b6-b1fe-eab26d379809 None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:02.427089 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cf25f5-6ac9-42b6-b1fe-eab26d379809 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:48:02.427362 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cf25f5-6ac9-42b6-b1fe-eab26d379809 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:02.428880 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cf25f5-6ac9-42b6-b1fe-eab26d379809 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:02.441963 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 612/1218] 10.222.0.23 () {56 vars in 1148 bytes} [Fri May 6 14:48:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:48:02.448026 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62bbcf7-39fa-4eb7-b05a-9ee497a1bc3c None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:02.563284 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62bbcf7-39fa-4eb7-b05a-9ee497a1bc3c None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:02.605413 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dbefe5-525b-4b7d-971f-2f5efdaac9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:02.608801 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dbefe5-525b-4b7d-971f-2f5efdaac9f7 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:02.609092 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dbefe5-525b-4b7d-971f-2f5efdaac9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:02.609374 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dbefe5-525b-4b7d-971f-2f5efdaac9f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:02.610205 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dbefe5-525b-4b7d-971f-2f5efdaac9f7 None admin] RBAC: Adding query filter params (role.id=f5641817ee56438496afc6135d37a4c0, scope.project.id=d5398feee6eb490cb49bb261448a1a9b, user.id=887a619ef5b5417db98ff15a9f712724) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:48:02.610518 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dbefe5-525b-4b7d-971f-2f5efdaac9f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:02.611962 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dbefe5-525b-4b7d-971f-2f5efdaac9f7 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:02.617821 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 607/1219] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:48:02 2022] GET /identity/v3/role_assignments?user.id=887a619ef5b5417db98ff15a9f712724&scope.project.id=d5398feee6eb490cb49bb261448a1a9b&role.id=f5641817ee56438496afc6135d37a4c0 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:48:05.008768 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc6b83-989e-4082-ad2a-8446bad6d4d7 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:05.009312 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc6b83-989e-4082-ad2a-8446bad6d4d7 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:05.009312 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc6b83-989e-4082-ad2a-8446bad6d4d7 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:05.010911 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 613/1220] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:48:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:48:05.017079 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643f88c-a259-425a-8dbe-9992a1264947 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:05.017623 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643f88c-a259-425a-8dbe-9992a1264947 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:05.017623 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643f88c-a259-425a-8dbe-9992a1264947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:05.034931 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-4643f88c-a259-425a-8dbe-9992a1264947 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:48:05.036049 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-4643f88c-a259-425a-8dbe-9992a1264947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:48:05.114253 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 608/1221] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:48:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:48:05.139491 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc6b83-989e-4082-ad2a-8446bad6d4d7 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:05.262028 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc6b83-989e-4082-ad2a-8446bad6d4d7 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:05.302487 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa06e339-1805-4521-ba96-daa3851e1259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:05.303285 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa06e339-1805-4521-ba96-daa3851e1259 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:05.303566 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa06e339-1805-4521-ba96-daa3851e1259 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:05.303837 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa06e339-1805-4521-ba96-daa3851e1259 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:05.315847 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa06e339-1805-4521-ba96-daa3851e1259 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:48:05.324999 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-aa06e339-1805-4521-ba96-daa3851e1259 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 06 14:48:05.324999 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:48:05.324999 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:48:05.324999 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:48:05.324999 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:48:05.324999 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:48:05.324999 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:48:05.324999 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:48:05.324999 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:48:05.324999 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:48:05.324999 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:48:05.324999 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:48:05.324999 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:48:05.324999 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:48:05.324999 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:48:05.324999 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:48:05.326560 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:48:05.326560 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:48:05.326560 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:48:05.326560 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:48:05.326560 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:48:05.326560 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:48:05.326560 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:48:05.326560 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:48:05.326560 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:48:05.326560 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:48:05.326560 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:48:05.326560 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:48:05.326560 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:48:05.326560 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:48:05.326560 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:48:05.326560 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:48:05.328190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:48:05.328190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:48:05.328190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:48:05.328190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:48:05.328190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:48:05.328190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:48:05.328190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:48:05.328190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:48:05.328190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:48:05.328190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:48:05.328190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 06 14:48:05.328190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:48:05.328190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 614/1222] 10.222.0.23 () {56 vars in 1102 bytes} [Fri May 6 14:48:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:48:05.333808 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4643f88c-a259-425a-8dbe-9992a1264947 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:05.444369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4643f88c-a259-425a-8dbe-9992a1264947 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:05.485850 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6644797f-4200-4212-9cbf-377767383cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:05.486551 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6644797f-4200-4212-9cbf-377767383cd2 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:05.486835 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6644797f-4200-4212-9cbf-377767383cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:05.487115 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6644797f-4200-4212-9cbf-377767383cd2 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:05.487869 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6644797f-4200-4212-9cbf-377767383cd2 None admin] RBAC: Adding query filter params (name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:48:05.488177 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6644797f-4200-4212-9cbf-377767383cd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:05.489572 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6644797f-4200-4212-9cbf-377767383cd2 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:05.498718 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 609/1223] 10.222.0.23 () {56 vars in 1103 bytes} [Fri May 6 14:48:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:48:05.504456 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa06e339-1805-4521-ba96-daa3851e1259 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:05.610408 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa06e339-1805-4521-ba96-daa3851e1259 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:05.646694 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88476371-42ea-4ee3-80b6-dad941240009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:05.647498 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88476371-42ea-4ee3-80b6-dad941240009 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:05.647821 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88476371-42ea-4ee3-80b6-dad941240009 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:05.648133 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88476371-42ea-4ee3-80b6-dad941240009 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:05.660509 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88476371-42ea-4ee3-80b6-dad941240009 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:05.662145 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88476371-42ea-4ee3-80b6-dad941240009 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:05.682973 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-88476371-42ea-4ee3-80b6-dad941240009 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:48:05.682973 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:48:05.682973 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:48:05.682973 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:48:05.682973 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:48:05.682973 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:48:05.682973 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:48:05.682973 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:48:05.682973 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:48:05.682973 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:48:05.682973 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:48:05.682973 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:48:05.682973 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:48:05.682973 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:48:05.682973 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:48:05.683956 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:48:05.683956 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:48:05.683956 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:48:05.683956 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:48:05.683956 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:48:05.683956 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:48:05.683956 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:48:05.683956 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:48:05.683956 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:48:05.683956 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:48:05.683956 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:48:05.683956 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:48:05.683956 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:48:05.683956 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:48:05.683956 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:48:05.683956 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:48:05.683956 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:48:05.684913 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:48:05.684913 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:48:05.684913 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:48:05.684913 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:48:05.688762 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 615/1224] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:48:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:48:05.690600 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6644797f-4200-4212-9cbf-377767383cd2 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:05.798166 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6644797f-4200-4212-9cbf-377767383cd2 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:05.837104 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4148a31-900e-47af-863f-885783b9ad30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:05.837769 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4148a31-900e-47af-863f-885783b9ad30 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:05.838047 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4148a31-900e-47af-863f-885783b9ad30 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:05.838378 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4148a31-900e-47af-863f-885783b9ad30 None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:05.839097 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4148a31-900e-47af-863f-885783b9ad30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:48:05.839400 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4148a31-900e-47af-863f-885783b9ad30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:05.840779 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4148a31-900e-47af-863f-885783b9ad30 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:05.861810 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 610/1225] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:48:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:48:05.863127 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88476371-42ea-4ee3-80b6-dad941240009 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:05.973249 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88476371-42ea-4ee3-80b6-dad941240009 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:06.006446 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975235c6-865e-412c-a55c-ae30b048b4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:06.007250 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975235c6-865e-412c-a55c-ae30b048b4aa None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:06.007536 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975235c6-865e-412c-a55c-ae30b048b4aa None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:06.007810 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975235c6-865e-412c-a55c-ae30b048b4aa None admin] PATH_INFO: `/v3/users/placement` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:06.015764 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975235c6-865e-412c-a55c-ae30b048b4aa None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:06.017236 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975235c6-865e-412c-a55c-ae30b048b4aa None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:06.031871 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-975235c6-865e-412c-a55c-ae30b048b4aa None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 06 14:48:06.031871 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:48:06.031871 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:48:06.031871 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:48:06.031871 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:48:06.031871 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:48:06.031871 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:48:06.031871 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:48:06.031871 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:48:06.031871 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:48:06.031871 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:48:06.031871 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:48:06.031871 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 06 14:48:06.031871 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_user(user_id) May 06 14:48:06.031871 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 06 14:48:06.031871 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 06 14:48:06.033339 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:48:06.033339 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:48:06.033339 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 06 14:48:06.033339 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:48:06.033339 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 06 14:48:06.033339 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:48:06.033339 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:48:06.033339 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:48:06.033339 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:48:06.033339 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:48:06.033339 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:48:06.033339 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:48:06.033339 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:48:06.033339 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:48:06.033339 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:48:06.033339 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:48:06.033339 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:48:06.033339 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:48:06.034500 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:48:06.034500 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:48:06.034500 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 06 14:48:06.034500 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 06 14:48:06.034500 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 06 14:48:06.034500 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 06 14:48:06.034500 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 06 14:48:06.034500 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 06 14:48:06.034500 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 06 14:48:06.034500 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:48:06.034500 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 616/1226] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:48:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:48:06.039215 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4148a31-900e-47af-863f-885783b9ad30 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:06.144865 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4148a31-900e-47af-863f-885783b9ad30 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:06.180804 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c2916a-59c0-4bee-9070-cb67fbb513c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:06.181530 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2916a-59c0-4bee-9070-cb67fbb513c8 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:06.181781 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2916a-59c0-4bee-9070-cb67fbb513c8 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:06.182055 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2916a-59c0-4bee-9070-cb67fbb513c8 None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:06.182883 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c2916a-59c0-4bee-9070-cb67fbb513c8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:48:06.183194 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c2916a-59c0-4bee-9070-cb67fbb513c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:06.184627 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c2916a-59c0-4bee-9070-cb67fbb513c8 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:06.202696 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 611/1227] 10.222.0.23 () {56 vars in 1143 bytes} [Fri May 6 14:48:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:48:06.208463 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975235c6-865e-412c-a55c-ae30b048b4aa None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:06.311482 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975235c6-865e-412c-a55c-ae30b048b4aa None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:06.345695 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82d9a3-b7bb-4beb-8f74-bc98d4a46fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:06.346490 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82d9a3-b7bb-4beb-8f74-bc98d4a46fd2 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:06.346830 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82d9a3-b7bb-4beb-8f74-bc98d4a46fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:06.347048 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82d9a3-b7bb-4beb-8f74-bc98d4a46fd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:06.358085 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f82d9a3-b7bb-4beb-8f74-bc98d4a46fd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:06.359523 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f82d9a3-b7bb-4beb-8f74-bc98d4a46fd2 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:06.376478 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-1f82d9a3-b7bb-4beb-8f74-bc98d4a46fd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:48:06.376478 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:48:06.376478 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:48:06.376478 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:48:06.376478 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:48:06.376478 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:48:06.376478 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:48:06.376478 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:48:06.376478 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:48:06.376478 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:48:06.376478 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:48:06.376478 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:48:06.376478 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:48:06.376478 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:48:06.376478 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:48:06.377611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:48:06.377611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:48:06.377611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:48:06.377611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:48:06.377611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:48:06.377611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:48:06.377611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:48:06.377611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:48:06.377611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:48:06.377611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:48:06.377611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:48:06.377611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:48:06.377611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:48:06.377611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:48:06.377611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:48:06.377611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:48:06.377611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:48:06.378819 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:48:06.378819 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:48:06.378819 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:48:06.378819 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:48:06.378819 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 617/1228] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:48:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:48:06.383484 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c2916a-59c0-4bee-9070-cb67fbb513c8 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:06.488282 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c2916a-59c0-4bee-9070-cb67fbb513c8 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:06.524130 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7b2000-8d70-4f30-a882-a69f4ffcc298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:06.524913 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b2000-8d70-4f30-a882-a69f4ffcc298 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:06.525198 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b2000-8d70-4f30-a882-a69f4ffcc298 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:06.525473 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b2000-8d70-4f30-a882-a69f4ffcc298 None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:06.526268 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7b2000-8d70-4f30-a882-a69f4ffcc298 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:48:06.526650 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7b2000-8d70-4f30-a882-a69f4ffcc298 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:06.528338 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7b2000-8d70-4f30-a882-a69f4ffcc298 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:06.547184 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 612/1229] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:48:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:48:06.548747 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82d9a3-b7bb-4beb-8f74-bc98d4a46fd2 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:06.656828 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82d9a3-b7bb-4beb-8f74-bc98d4a46fd2 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:06.692115 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19584491-e5f5-4de5-9b84-dd10ed481cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:06.692945 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19584491-e5f5-4de5-9b84-dd10ed481cf8 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:06.693226 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19584491-e5f5-4de5-9b84-dd10ed481cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:06.693496 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19584491-e5f5-4de5-9b84-dd10ed481cf8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:06.700019 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19584491-e5f5-4de5-9b84-dd10ed481cf8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:06.701535 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19584491-e5f5-4de5-9b84-dd10ed481cf8 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:06.715448 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-19584491-e5f5-4de5-9b84-dd10ed481cf8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:48:06.715448 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:48:06.715448 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:48:06.715448 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:48:06.715448 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:48:06.715448 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:48:06.715448 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:48:06.715448 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:48:06.715448 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:48:06.715448 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:48:06.715448 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:48:06.715448 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:48:06.715448 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 06 14:48:06.715448 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(project_id) May 06 14:48:06.715448 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 06 14:48:06.717356 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 06 14:48:06.717356 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:48:06.717356 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:48:06.717356 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:48:06.717356 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:48:06.717356 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:48:06.717356 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:48:06.717356 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:48:06.717356 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:48:06.717356 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:48:06.717356 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:48:06.717356 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:48:06.717356 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:48:06.717356 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:48:06.717356 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:48:06.717356 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:48:06.717356 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:48:06.719194 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 06 14:48:06.719194 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 06 14:48:06.719194 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 06 14:48:06.719194 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 06 14:48:06.719194 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 06 14:48:06.719194 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 06 14:48:06.719194 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:48:06.719194 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:48:06.719194 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 618/1230] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:48:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:48:06.723175 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7b2000-8d70-4f30-a882-a69f4ffcc298 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:06.825740 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7b2000-8d70-4f30-a882-a69f4ffcc298 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:06.864617 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df821abf-c538-44e4-9988-0c39902a0cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:06.865350 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df821abf-c538-44e4-9988-0c39902a0cb2 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:06.865628 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df821abf-c538-44e4-9988-0c39902a0cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:06.865911 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df821abf-c538-44e4-9988-0c39902a0cb2 None admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:06.866715 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df821abf-c538-44e4-9988-0c39902a0cb2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:48:06.867053 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df821abf-c538-44e4-9988-0c39902a0cb2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:06.868469 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df821abf-c538-44e4-9988-0c39902a0cb2 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:06.881974 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 613/1231] 10.222.0.23 () {56 vars in 1148 bytes} [Fri May 6 14:48:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:48:06.887974 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19584491-e5f5-4de5-9b84-dd10ed481cf8 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:06.991988 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19584491-e5f5-4de5-9b84-dd10ed481cf8 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:07.025160 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac7f90-ecca-433e-991c-10a83197b75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:07.026049 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac7f90-ecca-433e-991c-10a83197b75c None admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:07.026333 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac7f90-ecca-433e-991c-10a83197b75c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:07.026607 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac7f90-ecca-433e-991c-10a83197b75c None admin] PATH_INFO: `/v3/projects/d5398feee6eb490cb49bb261448a1a9b/users/887a619ef5b5417db98ff15a9f712724/roles/f5641817ee56438496afc6135d37a4c0` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:07.041249 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bac7f90-ecca-433e-991c-10a83197b75c None admin] RBAC: Authorizing `identity:create_grant(project_id=d5398feee6eb490cb49bb261448a1a9b, user_id=887a619ef5b5417db98ff15a9f712724, role_id=f5641817ee56438496afc6135d37a4c0)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:07.042715 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bac7f90-ecca-433e-991c-10a83197b75c None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:07.056489 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 619/1232] 10.222.0.23 () {58 vars in 1439 bytes} [Fri May 6 14:48:06 2022] PUT /identity/v3/projects/d5398feee6eb490cb49bb261448a1a9b/users/887a619ef5b5417db98ff15a9f712724/roles/f5641817ee56438496afc6135d37a4c0 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:48:09.293777 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eff2a2-54f8-4514-8044-ce870041c559 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:09.294368 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eff2a2-54f8-4514-8044-ce870041c559 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:09.294368 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eff2a2-54f8-4514-8044-ce870041c559 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:09.295712 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 614/1233] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:48:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:48:09.301309 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d683b4e-5925-4988-af6f-96cd08c2060f None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:09.301866 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d683b4e-5925-4988-af6f-96cd08c2060f None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:09.301969 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d683b4e-5925-4988-af6f-96cd08c2060f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:09.320103 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-8d683b4e-5925-4988-af6f-96cd08c2060f None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:48:09.321327 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-8d683b4e-5925-4988-af6f-96cd08c2060f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:48:09.396867 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 620/1234] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:48:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:48:09.421498 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eff2a2-54f8-4514-8044-ce870041c559 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:09.532119 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eff2a2-54f8-4514-8044-ce870041c559 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:09.572515 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c581d77-f319-4c30-b2b5-8b067a874add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:09.575049 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c581d77-f319-4c30-b2b5-8b067a874add None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:09.575049 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c581d77-f319-4c30-b2b5-8b067a874add None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:09.575049 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c581d77-f319-4c30-b2b5-8b067a874add None admin] PATH_INFO: `/v3/roles/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:09.588140 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c581d77-f319-4c30-b2b5-8b067a874add None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:48:09.598820 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-7c581d77-f319-4c30-b2b5-8b067a874add None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 06 14:48:09.598820 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:48:09.598820 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:48:09.598820 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:48:09.598820 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:48:09.598820 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:48:09.598820 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:48:09.598820 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:48:09.598820 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:48:09.598820 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:48:09.598820 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:48:09.598820 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:48:09.598820 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:48:09.598820 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:48:09.598820 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:48:09.598820 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:48:09.600089 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:48:09.600089 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:48:09.600089 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:48:09.600089 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:48:09.600089 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:48:09.600089 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:48:09.600089 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:48:09.600089 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:48:09.600089 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:48:09.600089 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:48:09.600089 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:48:09.600089 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:48:09.600089 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:48:09.600089 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:48:09.600089 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:48:09.600089 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:48:09.601143 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:48:09.601143 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:48:09.601143 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:48:09.601143 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:48:09.601143 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:48:09.601143 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:48:09.601143 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:48:09.601143 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:48:09.601143 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:48:09.601143 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:48:09.601143 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 06 14:48:09.601143 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:48:09.601143 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 615/1235] 10.222.0.23 () {56 vars in 1102 bytes} [Fri May 6 14:48:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:48:09.607524 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d683b4e-5925-4988-af6f-96cd08c2060f None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:09.711895 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d683b4e-5925-4988-af6f-96cd08c2060f None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:09.750174 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984c74b-b07f-4d7d-b726-2f353fddb3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:09.750856 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984c74b-b07f-4d7d-b726-2f353fddb3aa None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:09.751132 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984c74b-b07f-4d7d-b726-2f353fddb3aa None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:09.751399 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984c74b-b07f-4d7d-b726-2f353fddb3aa None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:09.752137 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0984c74b-b07f-4d7d-b726-2f353fddb3aa None admin] RBAC: Adding query filter params (name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:48:09.752438 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0984c74b-b07f-4d7d-b726-2f353fddb3aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:09.754078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0984c74b-b07f-4d7d-b726-2f353fddb3aa None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:09.762788 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 621/1236] 10.222.0.23 () {56 vars in 1103 bytes} [Fri May 6 14:48:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:48:09.769145 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c581d77-f319-4c30-b2b5-8b067a874add None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:09.876486 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c581d77-f319-4c30-b2b5-8b067a874add None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:09.914744 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165dfffd-23f6-4b0d-a4b4-9388dde7b3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:09.915563 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165dfffd-23f6-4b0d-a4b4-9388dde7b3e1 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:09.915907 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165dfffd-23f6-4b0d-a4b4-9388dde7b3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:09.916194 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165dfffd-23f6-4b0d-a4b4-9388dde7b3e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:09.927604 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165dfffd-23f6-4b0d-a4b4-9388dde7b3e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:09.929221 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165dfffd-23f6-4b0d-a4b4-9388dde7b3e1 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:09.947297 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-165dfffd-23f6-4b0d-a4b4-9388dde7b3e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:48:09.947297 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:48:09.947297 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:48:09.947297 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:48:09.947297 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:48:09.947297 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:48:09.947297 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:48:09.947297 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:48:09.947297 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:48:09.947297 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:48:09.947297 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:48:09.947297 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:48:09.947297 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:48:09.947297 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:48:09.947297 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:48:09.948582 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:48:09.948582 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:48:09.948582 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:48:09.948582 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:48:09.948582 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:48:09.948582 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:48:09.948582 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:48:09.948582 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:48:09.948582 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:48:09.948582 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:48:09.948582 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:48:09.948582 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:48:09.948582 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:48:09.948582 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:48:09.948582 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:48:09.948582 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:48:09.948582 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:48:09.949796 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:48:09.949796 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:48:09.949796 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:48:09.949796 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:48:09.949796 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 616/1237] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:48:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:48:09.955475 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984c74b-b07f-4d7d-b726-2f353fddb3aa None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:10.063358 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984c74b-b07f-4d7d-b726-2f353fddb3aa None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:10.099410 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cb8679-9c80-484e-beb8-50d0a0ce3c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:10.100101 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb8679-9c80-484e-beb8-50d0a0ce3c00 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:10.100381 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb8679-9c80-484e-beb8-50d0a0ce3c00 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:10.100676 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb8679-9c80-484e-beb8-50d0a0ce3c00 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:10.101458 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cb8679-9c80-484e-beb8-50d0a0ce3c00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:48:10.101798 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cb8679-9c80-484e-beb8-50d0a0ce3c00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:10.103200 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cb8679-9c80-484e-beb8-50d0a0ce3c00 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:10.119049 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 622/1238] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:48:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:48:10.126090 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165dfffd-23f6-4b0d-a4b4-9388dde7b3e1 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:10.236302 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165dfffd-23f6-4b0d-a4b4-9388dde7b3e1 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:10.275506 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f090f2-fff5-4a82-bd49-6f51dc2f4b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:10.276299 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f090f2-fff5-4a82-bd49-6f51dc2f4b4f None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:10.276575 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f090f2-fff5-4a82-bd49-6f51dc2f4b4f None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:10.276868 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f090f2-fff5-4a82-bd49-6f51dc2f4b4f None admin] PATH_INFO: `/v3/users/placement` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:10.285071 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f090f2-fff5-4a82-bd49-6f51dc2f4b4f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:10.286509 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f090f2-fff5-4a82-bd49-6f51dc2f4b4f None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:10.301725 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-53f090f2-fff5-4a82-bd49-6f51dc2f4b4f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 06 14:48:10.301725 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:48:10.301725 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:48:10.301725 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:48:10.301725 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:48:10.301725 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:48:10.301725 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:48:10.301725 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:48:10.301725 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:48:10.301725 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:48:10.301725 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:48:10.301725 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:48:10.301725 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 06 14:48:10.301725 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_user(user_id) May 06 14:48:10.301725 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 06 14:48:10.301725 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 06 14:48:10.303315 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:48:10.303315 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:48:10.303315 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 06 14:48:10.303315 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:48:10.303315 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 06 14:48:10.303315 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:48:10.303315 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:48:10.303315 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:48:10.303315 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:48:10.303315 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:48:10.303315 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:48:10.303315 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:48:10.303315 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:48:10.303315 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:48:10.303315 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:48:10.303315 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:48:10.303315 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:48:10.303315 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:48:10.304480 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:48:10.304480 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:48:10.304480 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 06 14:48:10.304480 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 06 14:48:10.304480 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 06 14:48:10.304480 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 06 14:48:10.304480 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 06 14:48:10.304480 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 06 14:48:10.304480 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 06 14:48:10.304480 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:48:10.304480 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 617/1239] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:48:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:48:10.309029 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cb8679-9c80-484e-beb8-50d0a0ce3c00 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:10.421270 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cb8679-9c80-484e-beb8-50d0a0ce3c00 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:10.455881 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86953d64-2ea1-4064-a36b-9a9305738780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:10.456589 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86953d64-2ea1-4064-a36b-9a9305738780 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:10.456894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86953d64-2ea1-4064-a36b-9a9305738780 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:10.457174 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86953d64-2ea1-4064-a36b-9a9305738780 None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:10.458000 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86953d64-2ea1-4064-a36b-9a9305738780 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:48:10.458304 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86953d64-2ea1-4064-a36b-9a9305738780 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:10.459730 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86953d64-2ea1-4064-a36b-9a9305738780 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:10.478194 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 623/1240] 10.222.0.23 () {56 vars in 1143 bytes} [Fri May 6 14:48:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:48:10.484317 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f090f2-fff5-4a82-bd49-6f51dc2f4b4f None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:10.592013 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f090f2-fff5-4a82-bd49-6f51dc2f4b4f None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:10.632110 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b5bcb6-3b9d-4f85-8b35-3a0181b688c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:10.632789 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5bcb6-3b9d-4f85-8b35-3a0181b688c6 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:10.633069 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5bcb6-3b9d-4f85-8b35-3a0181b688c6 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:10.633341 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5bcb6-3b9d-4f85-8b35-3a0181b688c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:10.707534 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b5bcb6-3b9d-4f85-8b35-3a0181b688c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:10.708982 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b5bcb6-3b9d-4f85-8b35-3a0181b688c6 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:10.812038 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 618/1241] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:48:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 06 14:48:10.818474 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86953d64-2ea1-4064-a36b-9a9305738780 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:10.929058 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86953d64-2ea1-4064-a36b-9a9305738780 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:10.967262 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f9cac-9e6a-4e5c-9abd-ec07eb0691de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:10.968091 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f9cac-9e6a-4e5c-9abd-ec07eb0691de None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:10.968414 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f9cac-9e6a-4e5c-9abd-ec07eb0691de None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:10.968754 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f9cac-9e6a-4e5c-9abd-ec07eb0691de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:10.981332 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00f9cac-9e6a-4e5c-9abd-ec07eb0691de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:10.983527 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00f9cac-9e6a-4e5c-9abd-ec07eb0691de None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:11.002273 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-e00f9cac-9e6a-4e5c-9abd-ec07eb0691de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:48:11.002273 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:48:11.002273 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:48:11.002273 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:48:11.002273 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:48:11.002273 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:48:11.002273 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:48:11.002273 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:48:11.002273 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:48:11.002273 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:48:11.002273 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:48:11.002273 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:48:11.002273 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:48:11.002273 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:48:11.002273 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:48:11.003249 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:48:11.003249 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:48:11.003249 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:48:11.003249 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:48:11.003249 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:48:11.003249 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:48:11.003249 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:48:11.003249 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:48:11.003249 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:48:11.003249 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:48:11.003249 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:48:11.003249 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:48:11.003249 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:48:11.003249 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:48:11.003249 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:48:11.003249 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:48:11.003249 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:48:11.004172 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:48:11.004172 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:48:11.004172 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:48:11.004172 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:48:11.004172 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 624/1242] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:48:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:48:11.009381 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b5bcb6-3b9d-4f85-8b35-3a0181b688c6 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:11.115230 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b5bcb6-3b9d-4f85-8b35-3a0181b688c6 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:11.152124 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a899a-513b-48fb-b9b4-7fdcd33868df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:11.152883 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a899a-513b-48fb-b9b4-7fdcd33868df None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:11.153218 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a899a-513b-48fb-b9b4-7fdcd33868df None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:11.154256 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a899a-513b-48fb-b9b4-7fdcd33868df None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:11.155350 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37a899a-513b-48fb-b9b4-7fdcd33868df None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:48:11.156109 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37a899a-513b-48fb-b9b4-7fdcd33868df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:11.157425 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37a899a-513b-48fb-b9b4-7fdcd33868df None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:11.172398 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 619/1243] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:48:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:48:11.178870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f9cac-9e6a-4e5c-9abd-ec07eb0691de None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:11.282133 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f9cac-9e6a-4e5c-9abd-ec07eb0691de None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:11.319974 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f2ba1-05de-4e59-9198-390143c5cb81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:11.320962 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f2ba1-05de-4e59-9198-390143c5cb81 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:11.321320 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f2ba1-05de-4e59-9198-390143c5cb81 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:11.321671 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f2ba1-05de-4e59-9198-390143c5cb81 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:11.329610 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319f2ba1-05de-4e59-9198-390143c5cb81 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:11.331351 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319f2ba1-05de-4e59-9198-390143c5cb81 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:11.348065 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-319f2ba1-05de-4e59-9198-390143c5cb81 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:48:11.348065 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:48:11.348065 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:48:11.348065 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:48:11.348065 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:48:11.348065 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:48:11.348065 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:48:11.348065 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:48:11.348065 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:48:11.348065 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:48:11.348065 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:48:11.348065 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:48:11.348065 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 06 14:48:11.348065 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(project_id) May 06 14:48:11.348065 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 06 14:48:11.349078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 06 14:48:11.349078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:48:11.349078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:48:11.349078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:48:11.349078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:48:11.349078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:48:11.349078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:48:11.349078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:48:11.349078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:48:11.349078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:48:11.349078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:48:11.349078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:48:11.349078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:48:11.349078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:48:11.349078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:48:11.349078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:48:11.349078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:48:11.350126 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 06 14:48:11.350126 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 06 14:48:11.350126 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 06 14:48:11.350126 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 06 14:48:11.350126 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 06 14:48:11.350126 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 06 14:48:11.350126 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:48:11.350126 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:48:11.350126 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 625/1244] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:48:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 06 14:48:11.355791 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a899a-513b-48fb-b9b4-7fdcd33868df None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:11.469608 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a899a-513b-48fb-b9b4-7fdcd33868df None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:11.507213 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ab99c-8751-416a-9c15-b4c05a281d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:11.507934 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ab99c-8751-416a-9c15-b4c05a281d8e None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:11.508299 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ab99c-8751-416a-9c15-b4c05a281d8e None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:11.508580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ab99c-8751-416a-9c15-b4c05a281d8e None admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:11.509518 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765ab99c-8751-416a-9c15-b4c05a281d8e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:48:11.509860 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765ab99c-8751-416a-9c15-b4c05a281d8e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:11.511425 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765ab99c-8751-416a-9c15-b4c05a281d8e None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:11.525762 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 620/1245] 10.222.0.23 () {56 vars in 1148 bytes} [Fri May 6 14:48:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:48:11.534456 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f2ba1-05de-4e59-9198-390143c5cb81 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:11.659347 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f2ba1-05de-4e59-9198-390143c5cb81 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:11.704357 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88a7f5b-ed93-4299-aa78-ae589aa6d99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:11.705036 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a7f5b-ed93-4299-aa78-ae589aa6d99d None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:11.705318 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a7f5b-ed93-4299-aa78-ae589aa6d99d None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:11.705589 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a7f5b-ed93-4299-aa78-ae589aa6d99d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:11.706484 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88a7f5b-ed93-4299-aa78-ae589aa6d99d None admin] RBAC: Adding query filter params (role.id=f5641817ee56438496afc6135d37a4c0, scope.project.id=d5398feee6eb490cb49bb261448a1a9b, user.id=887a619ef5b5417db98ff15a9f712724) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:48:11.706826 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88a7f5b-ed93-4299-aa78-ae589aa6d99d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:11.708389 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88a7f5b-ed93-4299-aa78-ae589aa6d99d None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:11.714309 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 626/1246] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:48:11 2022] GET /identity/v3/role_assignments?user.id=887a619ef5b5417db98ff15a9f712724&scope.project.id=d5398feee6eb490cb49bb261448a1a9b&role.id=f5641817ee56438496afc6135d37a4c0 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:48:14.111373 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f4151-de72-4a33-9f4d-90e339fdb6c4 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:14.111987 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f4151-de72-4a33-9f4d-90e339fdb6c4 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:14.111987 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f4151-de72-4a33-9f4d-90e339fdb6c4 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:14.113302 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 621/1247] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:48:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:48:14.118988 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff9921-83cf-4616-a52d-cfea8242d3b6 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:14.119434 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff9921-83cf-4616-a52d-cfea8242d3b6 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:14.119525 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff9921-83cf-4616-a52d-cfea8242d3b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:14.136352 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-65ff9921-83cf-4616-a52d-cfea8242d3b6 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:48:14.137652 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-65ff9921-83cf-4616-a52d-cfea8242d3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:48:14.205628 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 627/1248] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:48:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 06 14:48:14.231356 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f4151-de72-4a33-9f4d-90e339fdb6c4 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:14.351948 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f4151-de72-4a33-9f4d-90e339fdb6c4 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:14.388796 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41bd83-c874-4309-9405-c941f3da66ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:14.389719 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41bd83-c874-4309-9405-c941f3da66ae None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:14.389972 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41bd83-c874-4309-9405-c941f3da66ae None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:14.390281 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41bd83-c874-4309-9405-c941f3da66ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:14.402219 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad41bd83-c874-4309-9405-c941f3da66ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:48:14.412874 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-ad41bd83-c874-4309-9405-c941f3da66ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:48:14.412874 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:48:14.412874 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:48:14.412874 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:48:14.412874 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:48:14.412874 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:48:14.412874 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:48:14.412874 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:48:14.412874 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:48:14.412874 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:48:14.412874 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:48:14.412874 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:48:14.412874 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:48:14.412874 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:48:14.412874 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:48:14.412874 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:48:14.413843 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:48:14.413843 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:48:14.413843 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:48:14.413843 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:48:14.413843 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:48:14.413843 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:48:14.413843 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:48:14.413843 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:48:14.413843 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:48:14.413843 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:48:14.413843 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:48:14.413843 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:48:14.413843 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:48:14.413843 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:48:14.413843 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:48:14.413843 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:48:14.414692 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:48:14.414692 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:48:14.414692 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:48:14.414692 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:48:14.414692 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:48:14.414692 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:48:14.414692 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:48:14.414692 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:48:14.414692 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:48:14.414692 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:48:14.414692 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:48:14.414692 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:48:14.414692 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 622/1249] 10.222.0.23 () {56 vars in 1096 bytes} [Fri May 6 14:48:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:48:14.421308 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ff9921-83cf-4616-a52d-cfea8242d3b6 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:14.531503 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ff9921-83cf-4616-a52d-cfea8242d3b6 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:14.571202 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c8004-3c47-4980-bc9f-522c1ed6271f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:14.572115 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c8004-3c47-4980-bc9f-522c1ed6271f None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:14.572463 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c8004-3c47-4980-bc9f-522c1ed6271f None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:14.572903 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c8004-3c47-4980-bc9f-522c1ed6271f None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:14.573710 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276c8004-3c47-4980-bc9f-522c1ed6271f None admin] RBAC: Adding query filter params (name=admin) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:48:14.574041 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276c8004-3c47-4980-bc9f-522c1ed6271f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:14.576153 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276c8004-3c47-4980-bc9f-522c1ed6271f None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:14.585842 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 628/1250] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:48:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:48:14.592094 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41bd83-c874-4309-9405-c941f3da66ae None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:14.733258 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41bd83-c874-4309-9405-c941f3da66ae None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:14.768035 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c369e8-7bf1-4531-906b-d342b9145085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:14.768860 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c369e8-7bf1-4531-906b-d342b9145085 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:14.769145 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c369e8-7bf1-4531-906b-d342b9145085 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:14.769421 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c369e8-7bf1-4531-906b-d342b9145085 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:14.780834 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c369e8-7bf1-4531-906b-d342b9145085 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:14.782409 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c369e8-7bf1-4531-906b-d342b9145085 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:14.799171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-19c369e8-7bf1-4531-906b-d342b9145085 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:48:14.799171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:48:14.799171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:48:14.799171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:48:14.799171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:48:14.799171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:48:14.799171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:48:14.799171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:48:14.799171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:48:14.799171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:48:14.799171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:48:14.799171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:48:14.799171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:48:14.799171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:48:14.799171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:48:14.800131 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:48:14.800131 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:48:14.800131 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:48:14.800131 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:48:14.800131 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:48:14.800131 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:48:14.800131 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:48:14.800131 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:48:14.800131 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:48:14.800131 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:48:14.800131 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:48:14.800131 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:48:14.800131 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:48:14.800131 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:48:14.800131 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:48:14.800131 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:48:14.800131 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:48:14.801086 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:48:14.801086 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:48:14.801086 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:48:14.801086 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:48:14.801086 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 623/1251] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:48:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:48:14.806394 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c8004-3c47-4980-bc9f-522c1ed6271f None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:14.910573 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c8004-3c47-4980-bc9f-522c1ed6271f None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:14.944470 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d2dd52-7fc8-48cb-8a8c-dcb2599c6ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:14.945177 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2dd52-7fc8-48cb-8a8c-dcb2599c6ae3 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:14.945495 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2dd52-7fc8-48cb-8a8c-dcb2599c6ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:14.945763 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2dd52-7fc8-48cb-8a8c-dcb2599c6ae3 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:14.946484 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d2dd52-7fc8-48cb-8a8c-dcb2599c6ae3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:48:14.946796 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d2dd52-7fc8-48cb-8a8c-dcb2599c6ae3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:14.948150 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d2dd52-7fc8-48cb-8a8c-dcb2599c6ae3 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:14.960549 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 629/1252] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:48:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:48:14.966400 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c369e8-7bf1-4531-906b-d342b9145085 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:15.067553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c369e8-7bf1-4531-906b-d342b9145085 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:15.100396 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b769dba-c255-4e92-967f-197fd80b42bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:15.101452 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b769dba-c255-4e92-967f-197fd80b42bd None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:15.101737 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b769dba-c255-4e92-967f-197fd80b42bd None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:15.102018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b769dba-c255-4e92-967f-197fd80b42bd None admin] PATH_INFO: `/v3/users/placement` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:15.110512 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b769dba-c255-4e92-967f-197fd80b42bd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:15.111940 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b769dba-c255-4e92-967f-197fd80b42bd None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:15.126737 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-4b769dba-c255-4e92-967f-197fd80b42bd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 06 14:48:15.126737 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:48:15.126737 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:48:15.126737 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:48:15.126737 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:48:15.126737 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:48:15.126737 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:48:15.126737 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:48:15.126737 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:48:15.126737 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:48:15.126737 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:48:15.126737 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:48:15.126737 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 06 14:48:15.126737 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_user(user_id) May 06 14:48:15.126737 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 06 14:48:15.126737 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 06 14:48:15.127905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:48:15.127905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:48:15.127905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 06 14:48:15.127905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:48:15.127905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 06 14:48:15.127905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:48:15.127905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:48:15.127905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:48:15.127905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:48:15.127905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:48:15.127905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:48:15.127905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:48:15.127905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:48:15.127905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:48:15.127905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:48:15.127905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:48:15.127905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:48:15.127905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:48:15.129183 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:48:15.129183 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:48:15.129183 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 06 14:48:15.129183 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 06 14:48:15.129183 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 06 14:48:15.129183 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 06 14:48:15.129183 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 06 14:48:15.129183 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 06 14:48:15.129183 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 06 14:48:15.129183 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:48:15.132074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 624/1253] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:48:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:48:15.133788 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d2dd52-7fc8-48cb-8a8c-dcb2599c6ae3 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:15.244541 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d2dd52-7fc8-48cb-8a8c-dcb2599c6ae3 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:15.282318 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b280e50-c240-4cc8-ae12-5723bb00229a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:15.283004 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b280e50-c240-4cc8-ae12-5723bb00229a None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:15.283281 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b280e50-c240-4cc8-ae12-5723bb00229a None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:15.283550 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b280e50-c240-4cc8-ae12-5723bb00229a None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:15.284389 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b280e50-c240-4cc8-ae12-5723bb00229a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:48:15.284735 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b280e50-c240-4cc8-ae12-5723bb00229a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:15.286194 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b280e50-c240-4cc8-ae12-5723bb00229a None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:15.306610 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 630/1254] 10.222.0.23 () {56 vars in 1143 bytes} [Fri May 6 14:48:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:48:15.313588 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b769dba-c255-4e92-967f-197fd80b42bd None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:15.417647 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b769dba-c255-4e92-967f-197fd80b42bd None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:15.452507 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c81dc63-1ce2-4730-b633-462083418139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:15.453197 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c81dc63-1ce2-4730-b633-462083418139 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:15.453479 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c81dc63-1ce2-4730-b633-462083418139 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:15.453755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c81dc63-1ce2-4730-b633-462083418139 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:15.522284 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c81dc63-1ce2-4730-b633-462083418139 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:15.523753 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c81dc63-1ce2-4730-b633-462083418139 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:15.639194 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 625/1255] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:48:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 06 14:48:15.644988 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b280e50-c240-4cc8-ae12-5723bb00229a None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:15.743997 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b280e50-c240-4cc8-ae12-5723bb00229a None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:15.779994 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163abf5a-38c1-47f7-a867-63f222ff7fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:15.780850 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163abf5a-38c1-47f7-a867-63f222ff7fa8 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:15.781141 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163abf5a-38c1-47f7-a867-63f222ff7fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:15.781409 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163abf5a-38c1-47f7-a867-63f222ff7fa8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:15.792108 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163abf5a-38c1-47f7-a867-63f222ff7fa8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:15.793583 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163abf5a-38c1-47f7-a867-63f222ff7fa8 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:15.811088 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-163abf5a-38c1-47f7-a867-63f222ff7fa8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:48:15.811088 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:48:15.811088 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:48:15.811088 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:48:15.811088 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:48:15.811088 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:48:15.811088 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:48:15.811088 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:48:15.811088 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:48:15.811088 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:48:15.811088 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:48:15.811088 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:48:15.811088 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:48:15.811088 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:48:15.811088 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:48:15.812190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:48:15.812190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:48:15.812190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:48:15.812190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:48:15.812190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:48:15.812190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:48:15.812190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:48:15.812190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:48:15.812190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:48:15.812190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:48:15.812190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:48:15.812190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:48:15.812190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:48:15.812190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:48:15.812190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:48:15.812190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:48:15.812190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:48:15.813462 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:48:15.813462 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:48:15.813462 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:48:15.813462 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:48:15.813462 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 631/1256] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:48:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:48:15.819340 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c81dc63-1ce2-4730-b633-462083418139 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:15.917258 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c81dc63-1ce2-4730-b633-462083418139 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:15.948896 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1173cb52-812f-4bab-963f-1f1ae6d02e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:15.949548 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1173cb52-812f-4bab-963f-1f1ae6d02e29 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:15.949828 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1173cb52-812f-4bab-963f-1f1ae6d02e29 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:15.950102 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1173cb52-812f-4bab-963f-1f1ae6d02e29 None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:15.951459 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1173cb52-812f-4bab-963f-1f1ae6d02e29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:48:15.951765 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1173cb52-812f-4bab-963f-1f1ae6d02e29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:15.953165 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1173cb52-812f-4bab-963f-1f1ae6d02e29 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:15.963699 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 626/1257] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:48:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:48:15.970215 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163abf5a-38c1-47f7-a867-63f222ff7fa8 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:16.063011 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163abf5a-38c1-47f7-a867-63f222ff7fa8 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:16.095269 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3bc76b-af4d-413c-82b1-0c144471637e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:16.096038 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3bc76b-af4d-413c-82b1-0c144471637e None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:16.096316 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3bc76b-af4d-413c-82b1-0c144471637e None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:16.096581 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3bc76b-af4d-413c-82b1-0c144471637e None admin] PATH_INFO: `/v3/projects/service` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:16.102889 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3bc76b-af4d-413c-82b1-0c144471637e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:16.104313 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3bc76b-af4d-413c-82b1-0c144471637e None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:16.117479 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-1e3bc76b-af4d-413c-82b1-0c144471637e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:48:16.117479 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:48:16.117479 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:48:16.117479 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:48:16.117479 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:48:16.117479 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:48:16.117479 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:48:16.117479 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:48:16.117479 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:48:16.117479 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:48:16.117479 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:48:16.117479 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:48:16.117479 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 06 14:48:16.117479 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(project_id) May 06 14:48:16.117479 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 06 14:48:16.118529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 06 14:48:16.118529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:48:16.118529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:48:16.118529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:48:16.118529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:48:16.118529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:48:16.118529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:48:16.118529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:48:16.118529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:48:16.118529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:48:16.118529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:48:16.118529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:48:16.118529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:48:16.118529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:48:16.118529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:48:16.118529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:48:16.118529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:48:16.119793 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 06 14:48:16.119793 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 06 14:48:16.119793 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 06 14:48:16.119793 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 06 14:48:16.119793 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 06 14:48:16.119793 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 06 14:48:16.119793 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:48:16.119793 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:48:16.119793 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 632/1258] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:48:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:48:16.124909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1173cb52-812f-4bab-963f-1f1ae6d02e29 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:16.219378 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1173cb52-812f-4bab-963f-1f1ae6d02e29 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:16.250791 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393097a7-5ce0-4441-90b0-0d0e12893876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:16.251435 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393097a7-5ce0-4441-90b0-0d0e12893876 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:16.251713 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393097a7-5ce0-4441-90b0-0d0e12893876 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:16.251985 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393097a7-5ce0-4441-90b0-0d0e12893876 None admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:16.252780 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393097a7-5ce0-4441-90b0-0d0e12893876 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:48:16.253087 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393097a7-5ce0-4441-90b0-0d0e12893876 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:16.254494 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393097a7-5ce0-4441-90b0-0d0e12893876 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:16.266463 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 627/1259] 10.222.0.23 () {56 vars in 1148 bytes} [Fri May 6 14:48:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:48:16.272383 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3bc76b-af4d-413c-82b1-0c144471637e None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:16.366324 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3bc76b-af4d-413c-82b1-0c144471637e None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:16.398285 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aca0a1-7b03-4206-bb68-a95648d7a3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:16.398920 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aca0a1-7b03-4206-bb68-a95648d7a3db None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:16.399202 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aca0a1-7b03-4206-bb68-a95648d7a3db None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:16.399473 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aca0a1-7b03-4206-bb68-a95648d7a3db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:16.400281 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aca0a1-7b03-4206-bb68-a95648d7a3db None admin] RBAC: Adding query filter params (role.id=6dcd35d1a83748759d6ef9f026077835, scope.project.id=d5398feee6eb490cb49bb261448a1a9b, user.id=887a619ef5b5417db98ff15a9f712724) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:48:16.400588 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aca0a1-7b03-4206-bb68-a95648d7a3db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:16.402032 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aca0a1-7b03-4206-bb68-a95648d7a3db None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:16.407539 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 633/1260] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:48:16 2022] GET /identity/v3/role_assignments?user.id=887a619ef5b5417db98ff15a9f712724&scope.project.id=d5398feee6eb490cb49bb261448a1a9b&role.id=6dcd35d1a83748759d6ef9f026077835 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:48:18.632511 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875fe96d-eb92-4329-9e8e-a564a2af44ee None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:18.633077 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875fe96d-eb92-4329-9e8e-a564a2af44ee None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:18.633077 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875fe96d-eb92-4329-9e8e-a564a2af44ee None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:18.634448 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 628/1261] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:48:18.640216 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f540fe4-b57c-4438-9868-f83ac636751a None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:18.640676 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f540fe4-b57c-4438-9868-f83ac636751a None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:18.640797 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f540fe4-b57c-4438-9868-f83ac636751a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:18.657345 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-0f540fe4-b57c-4438-9868-f83ac636751a None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:48:18.658509 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-0f540fe4-b57c-4438-9868-f83ac636751a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:48:18.721163 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 634/1262] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:48:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:48:18.743765 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875fe96d-eb92-4329-9e8e-a564a2af44ee None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:18.840559 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875fe96d-eb92-4329-9e8e-a564a2af44ee None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:18.873088 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a70ee23-1260-4b00-a823-59870faed167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:18.873859 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70ee23-1260-4b00-a823-59870faed167 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:18.874139 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70ee23-1260-4b00-a823-59870faed167 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:18.874410 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70ee23-1260-4b00-a823-59870faed167 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:18.884079 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a70ee23-1260-4b00-a823-59870faed167 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:48:18.891639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-5a70ee23-1260-4b00-a823-59870faed167 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:48:18.891639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:48:18.891639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:48:18.891639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:48:18.891639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:48:18.891639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:48:18.891639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:48:18.891639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:48:18.891639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:48:18.891639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:48:18.891639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:48:18.891639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:48:18.891639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:48:18.891639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:48:18.891639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:48:18.891639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:48:18.892766 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:48:18.892766 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:48:18.892766 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:48:18.892766 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:48:18.892766 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:48:18.892766 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:48:18.892766 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:48:18.892766 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:48:18.892766 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:48:18.892766 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:48:18.892766 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:48:18.892766 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:48:18.892766 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:48:18.892766 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:48:18.892766 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:48:18.892766 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:48:18.893667 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:48:18.893667 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:48:18.893667 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:48:18.893667 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:48:18.893667 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:48:18.893667 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:48:18.893667 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:48:18.893667 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:48:18.893667 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:48:18.893667 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:48:18.893667 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:48:18.893667 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:48:18.893667 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 629/1263] 10.222.0.23 () {56 vars in 1096 bytes} [Fri May 6 14:48:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:48:18.898723 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f540fe4-b57c-4438-9868-f83ac636751a None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:18.994047 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f540fe4-b57c-4438-9868-f83ac636751a None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:19.026686 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947763c2-a66e-44e2-8711-eae8e4636357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:19.027340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947763c2-a66e-44e2-8711-eae8e4636357 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:19.027617 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947763c2-a66e-44e2-8711-eae8e4636357 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:19.027888 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947763c2-a66e-44e2-8711-eae8e4636357 None admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:19.029436 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947763c2-a66e-44e2-8711-eae8e4636357 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:48:19.029740 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947763c2-a66e-44e2-8711-eae8e4636357 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:19.031085 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947763c2-a66e-44e2-8711-eae8e4636357 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:19.038667 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 635/1264] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:48:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:48:19.044952 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a70ee23-1260-4b00-a823-59870faed167 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:19.139641 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a70ee23-1260-4b00-a823-59870faed167 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:19.174025 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4673f4c1-ceca-4430-b7a7-678c9a6e1420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:19.174809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673f4c1-ceca-4430-b7a7-678c9a6e1420 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:19.175087 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673f4c1-ceca-4430-b7a7-678c9a6e1420 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:19.175357 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673f4c1-ceca-4430-b7a7-678c9a6e1420 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:19.185372 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4673f4c1-ceca-4430-b7a7-678c9a6e1420 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:19.186793 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4673f4c1-ceca-4430-b7a7-678c9a6e1420 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:19.203001 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-4673f4c1-ceca-4430-b7a7-678c9a6e1420 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:48:19.203001 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:48:19.203001 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:48:19.203001 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:48:19.203001 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:48:19.203001 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:48:19.203001 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:48:19.203001 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:48:19.203001 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:48:19.203001 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:48:19.203001 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:48:19.203001 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:48:19.203001 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:48:19.203001 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:48:19.203001 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:48:19.204082 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:48:19.204082 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:48:19.204082 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:48:19.204082 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:48:19.204082 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:48:19.204082 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:48:19.204082 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:48:19.204082 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:48:19.204082 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:48:19.204082 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:48:19.204082 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:48:19.204082 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:48:19.204082 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:48:19.204082 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:48:19.204082 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:48:19.204082 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:48:19.204082 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:48:19.205394 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:48:19.205394 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:48:19.205394 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:48:19.205394 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:48:19.205394 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 630/1265] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:48:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:48:19.209865 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947763c2-a66e-44e2-8711-eae8e4636357 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:19.304165 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947763c2-a66e-44e2-8711-eae8e4636357 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:19.336347 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2436825b-1990-4699-9b9a-5bedcd2892e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:19.337033 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436825b-1990-4699-9b9a-5bedcd2892e9 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:19.337321 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436825b-1990-4699-9b9a-5bedcd2892e9 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:19.337600 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436825b-1990-4699-9b9a-5bedcd2892e9 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:19.338312 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2436825b-1990-4699-9b9a-5bedcd2892e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:48:19.338620 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2436825b-1990-4699-9b9a-5bedcd2892e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:19.339969 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2436825b-1990-4699-9b9a-5bedcd2892e9 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:19.351750 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 636/1266] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:48:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:48:19.357353 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4673f4c1-ceca-4430-b7a7-678c9a6e1420 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:19.454865 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4673f4c1-ceca-4430-b7a7-678c9a6e1420 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:19.486277 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d579cce4-3a59-4195-945d-b2f4f055d71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:19.487055 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579cce4-3a59-4195-945d-b2f4f055d71a None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:19.487332 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579cce4-3a59-4195-945d-b2f4f055d71a None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:19.487602 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579cce4-3a59-4195-945d-b2f4f055d71a None admin] PATH_INFO: `/v3/users/placement` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:19.494598 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d579cce4-3a59-4195-945d-b2f4f055d71a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:19.496015 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d579cce4-3a59-4195-945d-b2f4f055d71a None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:19.510293 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-d579cce4-3a59-4195-945d-b2f4f055d71a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 06 14:48:19.510293 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:48:19.510293 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:48:19.510293 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:48:19.510293 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:48:19.510293 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:48:19.510293 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:48:19.510293 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:48:19.510293 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:48:19.510293 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:48:19.510293 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:48:19.510293 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:48:19.510293 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 06 14:48:19.510293 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_user(user_id) May 06 14:48:19.510293 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 06 14:48:19.510293 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 06 14:48:19.511422 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:48:19.511422 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:48:19.511422 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 06 14:48:19.511422 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:48:19.511422 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 06 14:48:19.511422 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:48:19.511422 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:48:19.511422 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:48:19.511422 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:48:19.511422 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:48:19.511422 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:48:19.511422 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:48:19.511422 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:48:19.511422 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:48:19.511422 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:48:19.511422 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:48:19.511422 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:48:19.511422 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:48:19.512779 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:48:19.512779 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:48:19.512779 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 06 14:48:19.512779 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 06 14:48:19.512779 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 06 14:48:19.512779 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 06 14:48:19.512779 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 06 14:48:19.512779 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 06 14:48:19.512779 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 06 14:48:19.512779 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:48:19.512779 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 631/1267] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:48:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:48:19.517173 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2436825b-1990-4699-9b9a-5bedcd2892e9 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:19.620214 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2436825b-1990-4699-9b9a-5bedcd2892e9 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:19.653168 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89904934-55d6-4e80-9ff7-4ec0ca50926b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:19.655112 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89904934-55d6-4e80-9ff7-4ec0ca50926b None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:19.655361 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89904934-55d6-4e80-9ff7-4ec0ca50926b None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:19.655635 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89904934-55d6-4e80-9ff7-4ec0ca50926b None admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:19.656456 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89904934-55d6-4e80-9ff7-4ec0ca50926b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:48:19.656790 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89904934-55d6-4e80-9ff7-4ec0ca50926b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:19.658221 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89904934-55d6-4e80-9ff7-4ec0ca50926b None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:19.674926 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 637/1268] 10.222.0.23 () {56 vars in 1143 bytes} [Fri May 6 14:48:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:48:19.680676 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d579cce4-3a59-4195-945d-b2f4f055d71a None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:19.775440 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d579cce4-3a59-4195-945d-b2f4f055d71a None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:19.807202 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17002c3-ed96-460d-b235-c15cdc52a85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:19.807964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17002c3-ed96-460d-b235-c15cdc52a85f None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:19.808242 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17002c3-ed96-460d-b235-c15cdc52a85f None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:19.808512 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17002c3-ed96-460d-b235-c15cdc52a85f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:19.819188 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17002c3-ed96-460d-b235-c15cdc52a85f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:19.820624 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17002c3-ed96-460d-b235-c15cdc52a85f None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:19.836881 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-b17002c3-ed96-460d-b235-c15cdc52a85f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:48:19.836881 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:48:19.836881 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:48:19.836881 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:48:19.836881 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:48:19.836881 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:48:19.836881 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:48:19.836881 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:48:19.836881 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:48:19.836881 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:48:19.836881 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:48:19.836881 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:48:19.836881 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:48:19.836881 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:48:19.836881 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:48:19.837779 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:48:19.837779 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:48:19.837779 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:48:19.837779 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:48:19.837779 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:48:19.837779 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:48:19.837779 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:48:19.837779 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:48:19.837779 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:48:19.837779 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:48:19.837779 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:48:19.837779 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:48:19.837779 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:48:19.837779 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:48:19.837779 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:48:19.837779 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:48:19.837779 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:48:19.838830 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:48:19.838830 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:48:19.838830 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:48:19.838830 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:48:19.838830 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 632/1269] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:48:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:48:19.844137 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89904934-55d6-4e80-9ff7-4ec0ca50926b None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:19.937275 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89904934-55d6-4e80-9ff7-4ec0ca50926b None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:19.969201 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1283d9a-876d-4c09-8ee9-8c1f5ffd38b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:19.969848 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1283d9a-876d-4c09-8ee9-8c1f5ffd38b5 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:19.970134 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1283d9a-876d-4c09-8ee9-8c1f5ffd38b5 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:19.970408 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1283d9a-876d-4c09-8ee9-8c1f5ffd38b5 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:19.971111 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1283d9a-876d-4c09-8ee9-8c1f5ffd38b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:48:19.971415 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1283d9a-876d-4c09-8ee9-8c1f5ffd38b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:19.972786 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1283d9a-876d-4c09-8ee9-8c1f5ffd38b5 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:19.983581 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 638/1270] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:48:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:48:19.989135 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17002c3-ed96-460d-b235-c15cdc52a85f None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:20.084232 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17002c3-ed96-460d-b235-c15cdc52a85f None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:20.115122 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda22d8-dd62-43bf-a762-9e38e1939309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:20.115889 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda22d8-dd62-43bf-a762-9e38e1939309 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:20.116170 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda22d8-dd62-43bf-a762-9e38e1939309 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:20.116469 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda22d8-dd62-43bf-a762-9e38e1939309 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:20.123398 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda22d8-dd62-43bf-a762-9e38e1939309 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:20.124842 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda22d8-dd62-43bf-a762-9e38e1939309 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:20.139175 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-5dda22d8-dd62-43bf-a762-9e38e1939309 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:48:20.139175 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:48:20.139175 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:48:20.139175 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:48:20.139175 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:48:20.139175 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:48:20.139175 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:48:20.139175 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:48:20.139175 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:48:20.139175 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:48:20.139175 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:48:20.139175 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:48:20.139175 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 06 14:48:20.139175 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(project_id) May 06 14:48:20.139175 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 06 14:48:20.140062 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 06 14:48:20.140062 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:48:20.140062 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:48:20.140062 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:48:20.140062 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:48:20.140062 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:48:20.140062 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:48:20.140062 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:48:20.140062 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:48:20.140062 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:48:20.140062 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:48:20.140062 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:48:20.140062 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:48:20.140062 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:48:20.140062 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:48:20.140062 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:48:20.140062 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:48:20.141029 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 06 14:48:20.141029 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 06 14:48:20.141029 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 06 14:48:20.141029 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 06 14:48:20.141029 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 06 14:48:20.141029 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 06 14:48:20.141029 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:48:20.141029 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:48:20.141029 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 633/1271] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:48:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:48:20.146016 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1283d9a-876d-4c09-8ee9-8c1f5ffd38b5 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:20.240563 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1283d9a-876d-4c09-8ee9-8c1f5ffd38b5 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:20.272745 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82deb62-d751-4649-94bd-c519d2adf3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:20.273393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82deb62-d751-4649-94bd-c519d2adf3c5 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:20.273672 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82deb62-d751-4649-94bd-c519d2adf3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:20.273945 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82deb62-d751-4649-94bd-c519d2adf3c5 None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:20.274703 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82deb62-d751-4649-94bd-c519d2adf3c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:48:20.275005 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82deb62-d751-4649-94bd-c519d2adf3c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:20.276421 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82deb62-d751-4649-94bd-c519d2adf3c5 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:20.287953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 639/1272] 10.222.0.23 () {56 vars in 1148 bytes} [Fri May 6 14:48:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:48:20.294878 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda22d8-dd62-43bf-a762-9e38e1939309 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:20.389110 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda22d8-dd62-43bf-a762-9e38e1939309 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:20.421555 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7abeec-3d89-482a-8b3d-685196da30fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:20.422420 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7abeec-3d89-482a-8b3d-685196da30fe None admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:20.422696 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7abeec-3d89-482a-8b3d-685196da30fe None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:20.422965 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7abeec-3d89-482a-8b3d-685196da30fe None admin] PATH_INFO: `/v3/projects/d5398feee6eb490cb49bb261448a1a9b/users/887a619ef5b5417db98ff15a9f712724/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:20.425102 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7abeec-3d89-482a-8b3d-685196da30fe None admin] RBAC: Authorizing `identity:create_grant(project_id=d5398feee6eb490cb49bb261448a1a9b, user_id=887a619ef5b5417db98ff15a9f712724, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:20.426502 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7abeec-3d89-482a-8b3d-685196da30fe None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:20.439199 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 634/1273] 10.222.0.23 () {58 vars in 1439 bytes} [Fri May 6 14:48:20 2022] PUT /identity/v3/projects/d5398feee6eb490cb49bb261448a1a9b/users/887a619ef5b5417db98ff15a9f712724/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:48:22.730651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd16c18-8f94-42aa-b76f-d0c4f743b1fd None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:22.731184 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd16c18-8f94-42aa-b76f-d0c4f743b1fd None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:22.731184 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd16c18-8f94-42aa-b76f-d0c4f743b1fd None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:22.733082 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 640/1274] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:48:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:48:22.738502 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c627a94-c934-4b9c-b65d-4db568609297 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:22.738933 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c627a94-c934-4b9c-b65d-4db568609297 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:22.739064 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c627a94-c934-4b9c-b65d-4db568609297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:22.755797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-5c627a94-c934-4b9c-b65d-4db568609297 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:48:22.757007 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-5c627a94-c934-4b9c-b65d-4db568609297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:48:22.820771 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 635/1275] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:48:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 06 14:48:22.844781 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd16c18-8f94-42aa-b76f-d0c4f743b1fd None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:22.943208 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd16c18-8f94-42aa-b76f-d0c4f743b1fd None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:22.975810 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d56ec5-3636-449c-b799-f6b336d8cf79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:22.976585 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d56ec5-3636-449c-b799-f6b336d8cf79 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:22.977222 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d56ec5-3636-449c-b799-f6b336d8cf79 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:22.977499 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d56ec5-3636-449c-b799-f6b336d8cf79 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:22.988507 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d56ec5-3636-449c-b799-f6b336d8cf79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 06 14:48:22.996154 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-93d56ec5-3636-449c-b799-f6b336d8cf79 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:48:22.996154 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:48:22.996154 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:48:22.996154 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:48:22.996154 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:48:22.996154 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:48:22.996154 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:48:22.996154 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:48:22.996154 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:48:22.996154 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:48:22.996154 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:48:22.996154 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:48:22.996154 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 06 14:48:22.996154 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(role_id) May 06 14:48:22.996154 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 06 14:48:22.996154 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 06 14:48:22.997358 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 06 14:48:22.997358 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 06 14:48:22.997358 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 06 14:48:22.997358 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 06 14:48:22.997358 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:48:22.997358 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:48:22.997358 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:48:22.997358 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:48:22.997358 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:48:22.997358 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:48:22.997358 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:48:22.997358 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:48:22.997358 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:48:22.997358 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:48:22.997358 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:48:22.997358 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:48:22.998487 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:48:22.998487 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:48:22.998487 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:48:22.998487 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:48:22.998487 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 06 14:48:22.998487 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 06 14:48:22.998487 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 06 14:48:22.998487 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 06 14:48:22.998487 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 06 14:48:22.998487 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 06 14:48:22.998487 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 06 14:48:22.998487 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:48:22.998487 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 641/1276] 10.222.0.23 () {56 vars in 1096 bytes} [Fri May 6 14:48:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:48:23.003871 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c627a94-c934-4b9c-b65d-4db568609297 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:23.097151 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c627a94-c934-4b9c-b65d-4db568609297 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:23.129248 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04200b5-7a0d-4d2d-a4dc-38a6c47f4515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:23.129898 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04200b5-7a0d-4d2d-a4dc-38a6c47f4515 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:23.130181 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04200b5-7a0d-4d2d-a4dc-38a6c47f4515 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:23.130461 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04200b5-7a0d-4d2d-a4dc-38a6c47f4515 None admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:23.131193 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04200b5-7a0d-4d2d-a4dc-38a6c47f4515 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:48:23.131499 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04200b5-7a0d-4d2d-a4dc-38a6c47f4515 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:23.132884 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04200b5-7a0d-4d2d-a4dc-38a6c47f4515 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:23.140543 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 636/1277] 10.222.0.23 () {56 vars in 1099 bytes} [Fri May 6 14:48:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:48:23.146155 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d56ec5-3636-449c-b799-f6b336d8cf79 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:23.242458 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d56ec5-3636-449c-b799-f6b336d8cf79 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:23.273735 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d22d277-4db5-43b0-b8c3-89725d9fb975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:23.274506 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d22d277-4db5-43b0-b8c3-89725d9fb975 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:23.274787 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d22d277-4db5-43b0-b8c3-89725d9fb975 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:23.275061 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d22d277-4db5-43b0-b8c3-89725d9fb975 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:23.285780 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d22d277-4db5-43b0-b8c3-89725d9fb975 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:23.287206 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d22d277-4db5-43b0-b8c3-89725d9fb975 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:23.303733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-4d22d277-4db5-43b0-b8c3-89725d9fb975 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:48:23.303733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:48:23.303733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:48:23.303733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:48:23.303733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:48:23.303733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:48:23.303733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:48:23.303733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:48:23.303733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:48:23.303733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:48:23.303733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:48:23.303733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:48:23.303733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:48:23.303733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:48:23.303733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:48:23.304855 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:48:23.304855 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:48:23.304855 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:48:23.304855 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:48:23.304855 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:48:23.304855 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:48:23.304855 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:48:23.304855 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:48:23.304855 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:48:23.304855 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:48:23.304855 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:48:23.304855 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:48:23.304855 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:48:23.304855 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:48:23.304855 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:48:23.304855 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:48:23.304855 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:48:23.305800 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:48:23.305800 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:48:23.305800 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:48:23.305800 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:48:23.305800 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 642/1278] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:48:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:48:23.311247 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04200b5-7a0d-4d2d-a4dc-38a6c47f4515 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:23.404790 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04200b5-7a0d-4d2d-a4dc-38a6c47f4515 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:23.435380 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5331860f-94c7-4b3a-aa5d-4fd252d2fb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:23.436015 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331860f-94c7-4b3a-aa5d-4fd252d2fb6b None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:23.436294 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331860f-94c7-4b3a-aa5d-4fd252d2fb6b None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:23.436563 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331860f-94c7-4b3a-aa5d-4fd252d2fb6b None admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:23.437291 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5331860f-94c7-4b3a-aa5d-4fd252d2fb6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:48:23.437593 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5331860f-94c7-4b3a-aa5d-4fd252d2fb6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:23.438937 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5331860f-94c7-4b3a-aa5d-4fd252d2fb6b None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:23.450487 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 637/1279] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:48:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:48:23.456126 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d22d277-4db5-43b0-b8c3-89725d9fb975 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:23.555856 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d22d277-4db5-43b0-b8c3-89725d9fb975 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:23.592373 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb23ed-7cb8-407e-9f99-b973cb6fcc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:23.593211 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb23ed-7cb8-407e-9f99-b973cb6fcc95 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:23.593497 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb23ed-7cb8-407e-9f99-b973cb6fcc95 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:23.593761 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb23ed-7cb8-407e-9f99-b973cb6fcc95 None admin] PATH_INFO: `/v3/users/placement` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:23.601173 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bb23ed-7cb8-407e-9f99-b973cb6fcc95 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:23.602621 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bb23ed-7cb8-407e-9f99-b973cb6fcc95 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:23.617546 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-91bb23ed-7cb8-407e-9f99-b973cb6fcc95 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 06 14:48:23.617546 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:48:23.617546 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:48:23.617546 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:48:23.617546 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:48:23.617546 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:48:23.617546 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:48:23.617546 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:48:23.617546 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:48:23.617546 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:48:23.617546 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:48:23.617546 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:48:23.617546 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 06 14:48:23.617546 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_user(user_id) May 06 14:48:23.617546 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 06 14:48:23.617546 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 06 14:48:23.618737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:48:23.618737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:48:23.618737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 06 14:48:23.618737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:48:23.618737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 06 14:48:23.618737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 06 14:48:23.618737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:48:23.618737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:48:23.618737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:48:23.618737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:48:23.618737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:48:23.618737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:48:23.618737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:48:23.618737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:48:23.618737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:48:23.618737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:48:23.618737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:48:23.618737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:48:23.619935 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:48:23.619935 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:48:23.619935 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 06 14:48:23.619935 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 06 14:48:23.619935 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 06 14:48:23.619935 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 06 14:48:23.619935 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 06 14:48:23.619935 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 06 14:48:23.619935 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 06 14:48:23.619935 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:48:23.619935 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 643/1280] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:48:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:48:23.624834 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5331860f-94c7-4b3a-aa5d-4fd252d2fb6b None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:23.719521 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5331860f-94c7-4b3a-aa5d-4fd252d2fb6b None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:23.751239 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e102397f-6103-47a9-8662-f5bdc188d0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:23.751893 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102397f-6103-47a9-8662-f5bdc188d0a9 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:23.752174 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102397f-6103-47a9-8662-f5bdc188d0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:23.752446 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102397f-6103-47a9-8662-f5bdc188d0a9 None admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:23.753297 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e102397f-6103-47a9-8662-f5bdc188d0a9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:48:23.753602 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e102397f-6103-47a9-8662-f5bdc188d0a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:23.755010 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e102397f-6103-47a9-8662-f5bdc188d0a9 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:23.771084 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 638/1281] 10.222.0.23 () {56 vars in 1143 bytes} [Fri May 6 14:48:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:48:23.776773 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb23ed-7cb8-407e-9f99-b973cb6fcc95 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:23.871567 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb23ed-7cb8-407e-9f99-b973cb6fcc95 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:23.902869 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e94020-e58e-415c-bdc2-6fc77b5c7e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:23.904312 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e94020-e58e-415c-bdc2-6fc77b5c7e1c None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:23.904621 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e94020-e58e-415c-bdc2-6fc77b5c7e1c None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:23.904907 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e94020-e58e-415c-bdc2-6fc77b5c7e1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:23.971158 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e94020-e58e-415c-bdc2-6fc77b5c7e1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:23.972629 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e94020-e58e-415c-bdc2-6fc77b5c7e1c None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:24.070920 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 644/1282] 10.222.0.23 () {58 vars in 1282 bytes} [Fri May 6 14:48:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 06 14:48:24.076892 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e102397f-6103-47a9-8662-f5bdc188d0a9 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:24.171476 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e102397f-6103-47a9-8662-f5bdc188d0a9 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:24.203730 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5f8ae6-16ff-4396-8a86-4ecf6528ffb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:24.204518 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f8ae6-16ff-4396-8a86-4ecf6528ffb0 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:24.204832 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f8ae6-16ff-4396-8a86-4ecf6528ffb0 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:24.205112 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f8ae6-16ff-4396-8a86-4ecf6528ffb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:24.214610 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5f8ae6-16ff-4396-8a86-4ecf6528ffb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:24.216079 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5f8ae6-16ff-4396-8a86-4ecf6528ffb0 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:24.233833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-fc5f8ae6-16ff-4396-8a86-4ecf6528ffb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:48:24.233833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:48:24.233833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:48:24.233833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:48:24.233833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:48:24.233833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:48:24.233833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:48:24.233833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:48:24.233833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:48:24.233833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:48:24.233833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:48:24.233833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:48:24.233833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 06 14:48:24.233833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 06 14:48:24.233833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 06 14:48:24.234929 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 06 14:48:24.234929 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:48:24.234929 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:48:24.234929 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:48:24.234929 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:48:24.234929 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:48:24.234929 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:48:24.234929 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:48:24.234929 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:48:24.234929 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:48:24.234929 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:48:24.234929 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:48:24.234929 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:48:24.234929 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:48:24.234929 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:48:24.234929 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:48:24.234929 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:48:24.236031 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 06 14:48:24.236031 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 06 14:48:24.236031 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 06 14:48:24.236031 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:48:24.236031 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 639/1283] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:48:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:48:24.240642 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e94020-e58e-415c-bdc2-6fc77b5c7e1c None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:24.334586 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e94020-e58e-415c-bdc2-6fc77b5c7e1c None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:24.367317 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944b75d-9627-47e3-8242-10d1cee99d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:24.367976 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944b75d-9627-47e3-8242-10d1cee99d89 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:24.368260 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944b75d-9627-47e3-8242-10d1cee99d89 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:24.368535 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944b75d-9627-47e3-8242-10d1cee99d89 None admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:24.369270 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5944b75d-9627-47e3-8242-10d1cee99d89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:48:24.369575 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5944b75d-9627-47e3-8242-10d1cee99d89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:24.370925 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5944b75d-9627-47e3-8242-10d1cee99d89 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:24.381774 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 645/1284] 10.222.0.23 () {56 vars in 1109 bytes} [Fri May 6 14:48:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:48:24.388840 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5f8ae6-16ff-4396-8a86-4ecf6528ffb0 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:24.489911 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5f8ae6-16ff-4396-8a86-4ecf6528ffb0 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:24.527717 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654acc2-f8d8-41b1-b841-c959a8cb6519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:24.529728 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654acc2-f8d8-41b1-b841-c959a8cb6519 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:24.530010 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654acc2-f8d8-41b1-b841-c959a8cb6519 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:24.530283 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654acc2-f8d8-41b1-b841-c959a8cb6519 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:24.536889 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e654acc2-f8d8-41b1-b841-c959a8cb6519 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:24.538450 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e654acc2-f8d8-41b1-b841-c959a8cb6519 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:24.555711 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application [None req-e654acc2-f8d8-41b1-b841-c959a8cb6519 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:48:24.555711 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:48:24.555711 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:48:24.555711 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:48:24.555711 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:48:24.555711 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:48:24.555711 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:48:24.555711 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:48:24.555711 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:48:24.555711 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:48:24.555711 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:48:24.555711 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:48:24.555711 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 06 14:48:24.555711 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(project_id) May 06 14:48:24.555711 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 06 14:48:24.556862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 06 14:48:24.556862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:48:24.556862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:48:24.556862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:48:24.556862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:48:24.556862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:48:24.556862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:48:24.556862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:48:24.556862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application with Lock( May 06 14:48:24.556862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:48:24.556862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._enter() May 06 14:48:24.556862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:48:24.556862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:48:24.556862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:48:24.556862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.creator() May 06 14:48:24.556862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:48:24.556862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application created_value = creator( May 06 14:48:24.557872 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 06 14:48:24.557872 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 06 14:48:24.557872 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 06 14:48:24.557872 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 06 14:48:24.557872 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 06 14:48:24.557872 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 06 14:48:24.557872 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 06 14:48:24.557872 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: ERROR keystone.server.flask.application May 06 14:48:24.557872 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 640/1285] 10.222.0.23 () {56 vars in 1111 bytes} [Fri May 6 14:48:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:48:24.563796 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944b75d-9627-47e3-8242-10d1cee99d89 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:24.685427 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944b75d-9627-47e3-8242-10d1cee99d89 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:24.721308 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8e0737-ef43-4958-81db-7959180fbce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:24.722372 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e0737-ef43-4958-81db-7959180fbce9 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:24.722848 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e0737-ef43-4958-81db-7959180fbce9 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:24.723313 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e0737-ef43-4958-81db-7959180fbce9 None admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:24.724649 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8e0737-ef43-4958-81db-7959180fbce9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:48:24.725180 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8e0737-ef43-4958-81db-7959180fbce9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:24.727037 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8e0737-ef43-4958-81db-7959180fbce9 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:24.739214 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 646/1286] 10.222.0.23 () {56 vars in 1148 bytes} [Fri May 6 14:48:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:48:24.745479 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654acc2-f8d8-41b1-b841-c959a8cb6519 None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:24.851670 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654acc2-f8d8-41b1-b841-c959a8cb6519 None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:24.894140 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c84d9-181a-4d90-8645-ed9b6b4e5c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:24.894794 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c84d9-181a-4d90-8645-ed9b6b4e5c48 None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:24.895076 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c84d9-181a-4d90-8645-ed9b6b4e5c48 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:24.895352 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c84d9-181a-4d90-8645-ed9b6b4e5c48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:24.896169 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879c84d9-181a-4d90-8645-ed9b6b4e5c48 None admin] RBAC: Adding query filter params (role.id=6dcd35d1a83748759d6ef9f026077835, scope.project.id=d5398feee6eb490cb49bb261448a1a9b, user.id=887a619ef5b5417db98ff15a9f712724) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:48:24.896479 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879c84d9-181a-4d90-8645-ed9b6b4e5c48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:24.897934 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879c84d9-181a-4d90-8645-ed9b6b4e5c48 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:24.904687 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 641/1287] 10.222.0.23 () {56 vars in 1374 bytes} [Fri May 6 14:48:24 2022] GET /identity/v3/role_assignments?user.id=887a619ef5b5417db98ff15a9f712724&scope.project.id=d5398feee6eb490cb49bb261448a1a9b&role.id=6dcd35d1a83748759d6ef9f026077835 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:48:27.141415 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5e5c7-9748-4430-abd1-9027ddd70b9f None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:27.142010 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5e5c7-9748-4430-abd1-9027ddd70b9f None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:27.142010 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5e5c7-9748-4430-abd1-9027ddd70b9f None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:27.143352 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 647/1288] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:48:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:48:27.148916 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c0f58-545c-45c4-8a78-b1451f8f3751 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:27.149336 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c0f58-545c-45c4-8a78-b1451f8f3751 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:27.149476 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c0f58-545c-45c4-8a78-b1451f8f3751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:27.167245 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-9a9c0f58-545c-45c4-8a78-b1451f8f3751 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:48:27.168427 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-9a9c0f58-545c-45c4-8a78-b1451f8f3751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:48:27.230873 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 642/1289] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:48:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:48:27.253968 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5e5c7-9748-4430-abd1-9027ddd70b9f None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:27.348309 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5e5c7-9748-4430-abd1-9027ddd70b9f None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:27.379951 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d722d-2a05-461f-9695-b47db15be14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:27.380762 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d722d-2a05-461f-9695-b47db15be14e None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:27.381041 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d722d-2a05-461f-9695-b47db15be14e None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:27.381311 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d722d-2a05-461f-9695-b47db15be14e None admin] PATH_INFO: `/v3/services/placement` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:27.381953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613d722d-2a05-461f-9695-b47db15be14e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:27.383291 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613d722d-2a05-461f-9695-b47db15be14e None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:27.394851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-613d722d-2a05-461f-9695-b47db15be14e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 06 14:48:27.394851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:48:27.394851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 06 14:48:27.394851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 06 14:48:27.394851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 06 14:48:27.394851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 06 14:48:27.394851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 06 14:48:27.394851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:48:27.394851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 06 14:48:27.394851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:48:27.394851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 06 14:48:27.394851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:48:27.394851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:48:27.394851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:48:27.394851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:48:27.394851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:48:27.394851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:48:27.396031 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:48:27.396031 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:48:27.396031 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:48:27.396031 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:48:27.396031 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:48:27.396031 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:48:27.396031 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 06 14:48:27.396031 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(service_id) May 06 14:48:27.396031 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 06 14:48:27.396031 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 06 14:48:27.396031 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:48:27.396031 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:48:27.396031 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:48:27.396031 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:48:27.396031 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:48:27.396031 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:48:27.397147 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:48:27.397147 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:48:27.397147 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:48:27.397147 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:48:27.397147 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:48:27.397147 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:48:27.397147 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:48:27.397147 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:48:27.397147 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:48:27.397147 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:48:27.397147 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 06 14:48:27.397147 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:48:27.397147 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 06 14:48:27.397147 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:48:27.400755 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 648/1290] 10.222.0.23 () {56 vars in 1117 bytes} [Fri May 6 14:48:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:48:27.401924 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c0f58-545c-45c4-8a78-b1451f8f3751 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:27.495136 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c0f58-545c-45c4-8a78-b1451f8f3751 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:27.526590 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d0378-b71f-4e0d-bf6a-2fab883855ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:27.527236 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d0378-b71f-4e0d-bf6a-2fab883855ee None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:27.527516 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d0378-b71f-4e0d-bf6a-2fab883855ee None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:27.527789 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d0378-b71f-4e0d-bf6a-2fab883855ee None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:27.529299 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529d0378-b71f-4e0d-bf6a-2fab883855ee None admin] RBAC: Adding query filter params (name=placement) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:48:27.529698 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529d0378-b71f-4e0d-bf6a-2fab883855ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:27.531051 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529d0378-b71f-4e0d-bf6a-2fab883855ee None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:27.535642 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 643/1291] 10.222.0.23 () {56 vars in 1116 bytes} [Fri May 6 14:48:27 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:48:27.541329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d722d-2a05-461f-9695-b47db15be14e None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:27.645643 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d722d-2a05-461f-9695-b47db15be14e None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:27.679329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923f2997-2e79-49fa-9e66-d60e6b976d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:27.679982 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f2997-2e79-49fa-9e66-d60e6b976d12 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:27.680266 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f2997-2e79-49fa-9e66-d60e6b976d12 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:27.680539 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f2997-2e79-49fa-9e66-d60e6b976d12 None admin] PATH_INFO: `/v3/services` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:27.681266 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923f2997-2e79-49fa-9e66-d60e6b976d12 None admin] RBAC: Adding query filter params (type=placement) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:48:27.681569 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923f2997-2e79-49fa-9e66-d60e6b976d12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:27.682918 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923f2997-2e79-49fa-9e66-d60e6b976d12 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:27.687166 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 649/1292] 10.222.0.23 () {56 vars in 1116 bytes} [Fri May 6 14:48:27 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:48:29.896353 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e976dfe-69c9-4a3b-ab29-6d9440670c7d None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:29.896948 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e976dfe-69c9-4a3b-ab29-6d9440670c7d None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:29.896948 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e976dfe-69c9-4a3b-ab29-6d9440670c7d None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:29.898265 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 644/1293] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:48:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:48:29.903924 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c71a54-c45d-420c-a451-1f7ba1d33df6 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:29.904328 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c71a54-c45d-420c-a451-1f7ba1d33df6 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:29.904583 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c71a54-c45d-420c-a451-1f7ba1d33df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:29.921884 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-79c71a54-c45d-420c-a451-1f7ba1d33df6 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:48:29.923045 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-79c71a54-c45d-420c-a451-1f7ba1d33df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:48:29.988794 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 650/1294] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:48:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:48:30.010588 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e976dfe-69c9-4a3b-ab29-6d9440670c7d None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:30.106653 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e976dfe-69c9-4a3b-ab29-6d9440670c7d None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:30.140621 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d54d0-130b-44ef-b6bc-d7a24b52efd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:30.141388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d54d0-130b-44ef-b6bc-d7a24b52efd8 None admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:30.141682 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d54d0-130b-44ef-b6bc-d7a24b52efd8 None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:30.141957 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d54d0-130b-44ef-b6bc-d7a24b52efd8 None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:30.142693 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656d54d0-130b-44ef-b6bc-d7a24b52efd8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:30.144039 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656d54d0-130b-44ef-b6bc-d7a24b52efd8 None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:30.158098 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 645/1295] 10.222.0.23 () {60 vars in 1141 bytes} [Fri May 6 14:48:30 2022] POST /identity/v3/services => generated 246 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:48:32.401188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4ad34-6f71-4092-9723-dc8189273290 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:32.401748 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4ad34-6f71-4092-9723-dc8189273290 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:32.401748 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4ad34-6f71-4092-9723-dc8189273290 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:32.403404 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 651/1296] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:48:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:48:32.409160 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5297db-0a30-4787-b0f3-f9ccdfa4b430 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:32.409548 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5297db-0a30-4787-b0f3-f9ccdfa4b430 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:32.409651 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5297db-0a30-4787-b0f3-f9ccdfa4b430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:32.433253 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-ce5297db-0a30-4787-b0f3-f9ccdfa4b430 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:48:32.435033 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ce5297db-0a30-4787-b0f3-f9ccdfa4b430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:48:32.503585 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 646/1297] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:48:32 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 06 14:48:32.525822 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df4ad34-6f71-4092-9723-dc8189273290 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:32.623385 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df4ad34-6f71-4092-9723-dc8189273290 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:32.655034 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b793291-f272-4cee-8df6-859b4447d8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:32.655823 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b793291-f272-4cee-8df6-859b4447d8b5 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:32.656099 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b793291-f272-4cee-8df6-859b4447d8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:32.656367 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b793291-f272-4cee-8df6-859b4447d8b5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:32.657038 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b793291-f272-4cee-8df6-859b4447d8b5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:32.658386 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b793291-f272-4cee-8df6-859b4447d8b5 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:32.669934 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-5b793291-f272-4cee-8df6-859b4447d8b5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 06 14:48:32.669934 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:48:32.669934 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 06 14:48:32.669934 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 06 14:48:32.669934 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 06 14:48:32.669934 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 06 14:48:32.669934 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 06 14:48:32.669934 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:48:32.669934 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 06 14:48:32.669934 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:48:32.669934 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 06 14:48:32.669934 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:48:32.669934 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:48:32.669934 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:48:32.669934 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:48:32.669934 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:48:32.669934 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:48:32.671107 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:48:32.671107 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:48:32.671107 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:48:32.671107 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:48:32.671107 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:48:32.671107 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:48:32.671107 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 06 14:48:32.671107 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(service_id) May 06 14:48:32.671107 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 06 14:48:32.671107 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 06 14:48:32.671107 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:48:32.671107 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:48:32.671107 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:48:32.671107 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:48:32.671107 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:48:32.671107 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:48:32.672359 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:48:32.672359 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:48:32.672359 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:48:32.672359 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:48:32.672359 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:48:32.672359 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:48:32.672359 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:48:32.672359 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:48:32.672359 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:48:32.672359 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:48:32.672359 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 06 14:48:32.672359 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:48:32.672359 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 06 14:48:32.672359 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:48:32.672359 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 652/1298] 10.222.0.23 () {56 vars in 1117 bytes} [Fri May 6 14:48:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:48:32.677138 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5297db-0a30-4787-b0f3-f9ccdfa4b430 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:32.771054 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5297db-0a30-4787-b0f3-f9ccdfa4b430 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:32.802843 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d6cacd-68a4-4be7-a77e-15373e0ff00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:32.803494 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6cacd-68a4-4be7-a77e-15373e0ff00f None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:32.803774 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6cacd-68a4-4be7-a77e-15373e0ff00f None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:32.804043 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6cacd-68a4-4be7-a77e-15373e0ff00f None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:32.804759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d6cacd-68a4-4be7-a77e-15373e0ff00f None admin] RBAC: Adding query filter params (name=placement) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:48:32.805063 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d6cacd-68a4-4be7-a77e-15373e0ff00f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:32.806424 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d6cacd-68a4-4be7-a77e-15373e0ff00f None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:32.811770 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 647/1299] 10.222.0.23 () {56 vars in 1116 bytes} [Fri May 6 14:48:32 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:48:32.817752 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b793291-f272-4cee-8df6-859b4447d8b5 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:32.914837 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b793291-f272-4cee-8df6-859b4447d8b5 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:32.946643 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b0375-a8f6-4d78-9574-2bac0f6d884b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:32.947222 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b0375-a8f6-4d78-9574-2bac0f6d884b None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:32.947505 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b0375-a8f6-4d78-9574-2bac0f6d884b None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:32.947777 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b0375-a8f6-4d78-9574-2bac0f6d884b None admin] PATH_INFO: `/v3/endpoints` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:32.948545 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72b0375-a8f6-4d78-9574-2bac0f6d884b None admin] RBAC: Adding query filter params (interface=public, service_id=c183cd7a999c4ca3978778d05b401531, region_id=RegionOne) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:48:32.948880 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72b0375-a8f6-4d78-9574-2bac0f6d884b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:32.950804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72b0375-a8f6-4d78-9574-2bac0f6d884b None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:32.955709 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 653/1300] 10.222.0.23 () {56 vars in 1251 bytes} [Fri May 6 14:48:32 2022] GET /identity/v3/endpoints?service_id=c183cd7a999c4ca3978778d05b401531&interface=public®ion_id=RegionOne => generated 194 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:48:32.961484 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d6cacd-68a4-4be7-a77e-15373e0ff00f None admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:33.057361 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d6cacd-68a4-4be7-a77e-15373e0ff00f None admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:33.090260 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2621f5-3b3f-4534-a1e3-7997f772ccdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:33.091383 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2621f5-3b3f-4534-a1e3-7997f772ccdc None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:33.091664 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2621f5-3b3f-4534-a1e3-7997f772ccdc None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:33.091940 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2621f5-3b3f-4534-a1e3-7997f772ccdc None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:33.092636 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2621f5-3b3f-4534-a1e3-7997f772ccdc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:33.093982 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2621f5-3b3f-4534-a1e3-7997f772ccdc None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:33.100429 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 648/1301] 10.222.0.23 () {56 vars in 1087 bytes} [Fri May 6 14:48:32 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 06 14:48:35.343022 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681a91b-c66e-45e8-9c16-1e273f8debac None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:35.343544 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681a91b-c66e-45e8-9c16-1e273f8debac None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:35.343544 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681a91b-c66e-45e8-9c16-1e273f8debac None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:35.344945 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 654/1302] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:48:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:48:35.350070 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af60ea7-4608-4d85-b4a1-36006c44ae08 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:35.350489 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af60ea7-4608-4d85-b4a1-36006c44ae08 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:35.350632 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af60ea7-4608-4d85-b4a1-36006c44ae08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:35.366974 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-6af60ea7-4608-4d85-b4a1-36006c44ae08 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:48:35.368149 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-6af60ea7-4608-4d85-b4a1-36006c44ae08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:48:35.431130 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 649/1303] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:48:35 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 06 14:48:35.454382 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5681a91b-c66e-45e8-9c16-1e273f8debac None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:35.550513 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5681a91b-c66e-45e8-9c16-1e273f8debac None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:35.582304 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63642d38-cf40-4abc-9c72-ad62268d9eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:35.583079 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63642d38-cf40-4abc-9c72-ad62268d9eb0 None admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:35.583358 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63642d38-cf40-4abc-9c72-ad62268d9eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:35.583629 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63642d38-cf40-4abc-9c72-ad62268d9eb0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:35.584257 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63642d38-cf40-4abc-9c72-ad62268d9eb0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:35.585637 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63642d38-cf40-4abc-9c72-ad62268d9eb0 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:35.597739 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application [None req-63642d38-cf40-4abc-9c72-ad62268d9eb0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 06 14:48:35.597739 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:48:35.597739 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 06 14:48:35.597739 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 06 14:48:35.597739 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 06 14:48:35.597739 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 06 14:48:35.597739 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 06 14:48:35.597739 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:48:35.597739 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 06 14:48:35.597739 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:48:35.597739 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 06 14:48:35.597739 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:48:35.597739 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application Traceback (most recent call last): May 06 14:48:35.597739 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 06 14:48:35.597739 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 06 14:48:35.597739 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 06 14:48:35.597739 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 06 14:48:35.598909 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 06 14:48:35.598909 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 06 14:48:35.598909 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 06 14:48:35.598909 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 06 14:48:35.598909 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 06 14:48:35.598909 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 06 14:48:35.598909 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 06 14:48:35.598909 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._get_service(service_id) May 06 14:48:35.598909 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 06 14:48:35.598909 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 06 14:48:35.598909 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 06 14:48:35.598909 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 06 14:48:35.598909 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 06 14:48:35.598909 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 06 14:48:35.598909 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 06 14:48:35.598909 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.get_or_create( May 06 14:48:35.599946 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 06 14:48:35.599946 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application with Lock( May 06 14:48:35.599946 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 06 14:48:35.599946 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self._enter() May 06 14:48:35.599946 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 06 14:48:35.599946 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 06 14:48:35.599946 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 06 14:48:35.599946 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application return self.creator() May 06 14:48:35.599946 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 06 14:48:35.599946 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application created_value = creator( May 06 14:48:35.599946 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 06 14:48:35.599946 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 06 14:48:35.599946 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 06 14:48:35.599946 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: ERROR keystone.server.flask.application May 06 14:48:35.599946 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 655/1304] 10.222.0.23 () {56 vars in 1117 bytes} [Fri May 6 14:48:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 06 14:48:35.604862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af60ea7-4608-4d85-b4a1-36006c44ae08 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:35.700454 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af60ea7-4608-4d85-b4a1-36006c44ae08 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:35.736393 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20cebdd-7883-4398-9d58-c5265b96f86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:35.737089 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20cebdd-7883-4398-9d58-c5265b96f86f None admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:35.737373 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20cebdd-7883-4398-9d58-c5265b96f86f None admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:35.737644 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20cebdd-7883-4398-9d58-c5265b96f86f None admin] PATH_INFO: `/v3/services` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:35.738354 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20cebdd-7883-4398-9d58-c5265b96f86f None admin] RBAC: Adding query filter params (name=placement) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:48:35.738663 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20cebdd-7883-4398-9d58-c5265b96f86f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:35.740115 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20cebdd-7883-4398-9d58-c5265b96f86f None admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:35.746086 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 650/1305] 10.222.0.23 () {56 vars in 1116 bytes} [Fri May 6 14:48:35 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:48:35.752839 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63642d38-cf40-4abc-9c72-ad62268d9eb0 None admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:48:35.852564 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63642d38-cf40-4abc-9c72-ad62268d9eb0 None admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:48:35.886544 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c8b9b9-a53e-498c-bb74-849af67b9404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:48:35.887239 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c8b9b9-a53e-498c-bb74-849af67b9404 None admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:48:35.888623 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c8b9b9-a53e-498c-bb74-849af67b9404 None admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:48:35.888906 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c8b9b9-a53e-498c-bb74-849af67b9404 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:48:35.889628 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c8b9b9-a53e-498c-bb74-849af67b9404 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:48:35.890975 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c8b9b9-a53e-498c-bb74-849af67b9404 None admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:48:35.908984 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 656/1306] 10.222.0.23 () {60 vars in 1144 bytes} [Fri May 6 14:48:35 2022] POST /identity/v3/endpoints => generated 331 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:49:40.236114 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad4030-7402-4f5f-b01e-8b7b5bb7e5b6 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:49:40.238267 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad4030-7402-4f5f-b01e-8b7b5bb7e5b6 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:49:40.239467 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad4030-7402-4f5f-b01e-8b7b5bb7e5b6 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:49:40.250295 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe26f8-7478-40b5-b09c-d3800403f707 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:49:40.250717 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe26f8-7478-40b5-b09c-d3800403f707 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:49:40.250717 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe26f8-7478-40b5-b09c-d3800403f707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:49:40.252002 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 651/1307] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:49:40 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:49:40.437590 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-30fe26f8-7478-40b5-b09c-d3800403f707 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:49:40.439491 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-30fe26f8-7478-40b5-b09c-d3800403f707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:49:40.628635 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 657/1308] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:49:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:49:40.666638 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9702b7e-0181-453d-a7cb-555a756af9af None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:49:40.667091 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9702b7e-0181-453d-a7cb-555a756af9af None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:49:40.667486 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9702b7e-0181-453d-a7cb-555a756af9af None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:49:40.669284 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 652/1309] 10.222.0.23 () {54 vars in 964 bytes} [Fri May 6 14:49:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:49:40.675609 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ead6f-e0e3-4b51-9574-9639c5601bce None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:49:40.675906 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ead6f-e0e3-4b51-9574-9639c5601bce None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:49:40.676188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ead6f-e0e3-4b51-9574-9639c5601bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:49:40.762583 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-2f6ead6f-e0e3-4b51-9574-9639c5601bce None None] MFA Rules not processed for user `4dcd1542d8f643c4996142a88b9b6abb`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:49:40.764011 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-2f6ead6f-e0e3-4b51-9574-9639c5601bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:49:40.858609 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 658/1310] 10.222.0.23 () {58 vars in 1061 bytes} [Fri May 6 14:49:40 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:49:40.868747 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9702b7e-0181-453d-a7cb-555a756af9af None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:49:40.870990 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-e9702b7e-0181-453d-a7cb-555a756af9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:49:40.989938 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9702b7e-0181-453d-a7cb-555a756af9af None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:49:41.079960 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1132a6-72c3-4bba-b104-b04ea4bd4d6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:49:41.081184 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1132a6-72c3-4bba-b104-b04ea4bd4d6e service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:49:41.081719 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1132a6-72c3-4bba-b104-b04ea4bd4d6e service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:49:41.082317 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1132a6-72c3-4bba-b104-b04ea4bd4d6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:49:41.084154 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-8f1132a6-72c3-4bba-b104-b04ea4bd4d6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:49:41.208530 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1132a6-72c3-4bba-b104-b04ea4bd4d6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:49:41.210372 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1132a6-72c3-4bba-b104-b04ea4bd4d6e service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:49:41.235102 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 653/1311] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:49:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:49:44.069904 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca3d09f-96ae-4df3-8956-1574df1ec3cb None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:49:44.070429 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca3d09f-96ae-4df3-8956-1574df1ec3cb None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:49:44.070429 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca3d09f-96ae-4df3-8956-1574df1ec3cb None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:49:44.071856 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 659/1312] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:49:44.077655 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be11f35-25c9-4c18-8ace-1ae27d9eb5d2 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:49:44.078085 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be11f35-25c9-4c18-8ace-1ae27d9eb5d2 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:49:44.078403 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be11f35-25c9-4c18-8ace-1ae27d9eb5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:49:44.152351 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-3be11f35-25c9-4c18-8ace-1ae27d9eb5d2 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:49:44.153800 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-3be11f35-25c9-4c18-8ace-1ae27d9eb5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:49:44.260752 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 654/1313] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:49:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:49:44.300670 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca3d09f-96ae-4df3-8956-1574df1ec3cb None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:49:44.302216 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-eca3d09f-96ae-4df3-8956-1574df1ec3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:49:44.408157 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca3d09f-96ae-4df3-8956-1574df1ec3cb None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:49:44.480547 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e02d03-49e8-40ea-98bf-b9cb18684427 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:49:44.481228 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e02d03-49e8-40ea-98bf-b9cb18684427 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:49:44.481512 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e02d03-49e8-40ea-98bf-b9cb18684427 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:49:44.481791 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e02d03-49e8-40ea-98bf-b9cb18684427 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:49:44.483409 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-b7e02d03-49e8-40ea-98bf-b9cb18684427 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:49:44.593771 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e02d03-49e8-40ea-98bf-b9cb18684427 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:49:44.595371 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e02d03-49e8-40ea-98bf-b9cb18684427 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:49:44.616090 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 660/1314] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:49:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:49:47.215189 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07ddb4-08fe-4fa3-b020-859e40cdb660 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:49:47.215706 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07ddb4-08fe-4fa3-b020-859e40cdb660 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:49:47.215800 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07ddb4-08fe-4fa3-b020-859e40cdb660 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:49:47.217184 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 655/1315] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:49:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:49:47.222525 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec305e-5eda-4f28-af8d-9ad1dfecd79f None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:49:47.223033 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec305e-5eda-4f28-af8d-9ad1dfecd79f None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:49:47.223141 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec305e-5eda-4f28-af8d-9ad1dfecd79f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:49:47.287193 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-64ec305e-5eda-4f28-af8d-9ad1dfecd79f None None] MFA Rules not processed for user `f26c0bc8ce6d4932bba691c04bec3ded`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:49:47.288459 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-64ec305e-5eda-4f28-af8d-9ad1dfecd79f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:49:47.385066 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 661/1316] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:49:47 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:49:47.440979 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59904458-5f2e-41c6-8819-87935b9d318c None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:49:47.441265 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59904458-5f2e-41c6-8819-87935b9d318c None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:49:47.441580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59904458-5f2e-41c6-8819-87935b9d318c None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:49:47.443041 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 656/1317] 10.222.0.23 () {54 vars in 964 bytes} [Fri May 6 14:49:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:49:47.449879 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de937a75-ff7d-467d-b628-e895aabee068 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:49:47.450172 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de937a75-ff7d-467d-b628-e895aabee068 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:49:47.450471 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de937a75-ff7d-467d-b628-e895aabee068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:49:47.522491 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-de937a75-ff7d-467d-b628-e895aabee068 None None] MFA Rules not processed for user `4dcd1542d8f643c4996142a88b9b6abb`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:49:47.523698 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-de937a75-ff7d-467d-b628-e895aabee068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:49:47.604303 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 662/1318] 10.222.0.23 () {58 vars in 1061 bytes} [Fri May 6 14:49:47 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:49:47.613290 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59904458-5f2e-41c6-8819-87935b9d318c None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:49:47.614797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-59904458-5f2e-41c6-8819-87935b9d318c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:49:47.716360 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59904458-5f2e-41c6-8819-87935b9d318c None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:49:47.787429 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd35e39a-d8a9-48c8-81f6-7db56577bd02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:49:47.788082 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35e39a-d8a9-48c8-81f6-7db56577bd02 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:49:47.788366 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35e39a-d8a9-48c8-81f6-7db56577bd02 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:49:47.788663 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35e39a-d8a9-48c8-81f6-7db56577bd02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:49:47.790332 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-bd35e39a-d8a9-48c8-81f6-7db56577bd02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:49:47.905989 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd35e39a-d8a9-48c8-81f6-7db56577bd02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:49:47.907477 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd35e39a-d8a9-48c8-81f6-7db56577bd02 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:49:47.925609 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 657/1319] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:49:47 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:49:51.109971 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f082730b-b366-4691-b89f-beacbb26a6ef None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:49:51.110510 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f082730b-b366-4691-b89f-beacbb26a6ef None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:49:51.110510 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f082730b-b366-4691-b89f-beacbb26a6ef None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:49:51.111894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 663/1320] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:49:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:49:51.117492 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab596d-cf06-4a51-a30f-ec765268c2a2 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:49:51.118627 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab596d-cf06-4a51-a30f-ec765268c2a2 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:49:51.118627 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab596d-cf06-4a51-a30f-ec765268c2a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:49:51.182872 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-cdab596d-cf06-4a51-a30f-ec765268c2a2 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:49:51.184085 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-cdab596d-cf06-4a51-a30f-ec765268c2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:49:51.276695 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 658/1321] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:49:51 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:49:51.316245 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f082730b-b366-4691-b89f-beacbb26a6ef None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:49:51.317825 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f082730b-b366-4691-b89f-beacbb26a6ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:49:51.418442 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f082730b-b366-4691-b89f-beacbb26a6ef None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:49:51.488008 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c22f25e-7ec3-4d67-a5b9-d65319197cf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:49:51.488688 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22f25e-7ec3-4d67-a5b9-d65319197cf0 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:49:51.489243 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22f25e-7ec3-4d67-a5b9-d65319197cf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:49:51.489526 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22f25e-7ec3-4d67-a5b9-d65319197cf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:49:51.491173 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-3c22f25e-7ec3-4d67-a5b9-d65319197cf0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:49:51.601273 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c22f25e-7ec3-4d67-a5b9-d65319197cf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:49:51.602881 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c22f25e-7ec3-4d67-a5b9-d65319197cf0 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:49:51.620585 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 664/1322] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:49:51 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:49:54.636139 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd504935-90c5-464a-a13c-9794560e415c None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:49:54.636703 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd504935-90c5-464a-a13c-9794560e415c None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:49:54.636703 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd504935-90c5-464a-a13c-9794560e415c None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:49:54.638088 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 659/1323] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:49:54.643476 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d680a5-5ef1-40fc-b270-b7fd5de4abd6 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:49:54.643888 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d680a5-5ef1-40fc-b270-b7fd5de4abd6 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:49:54.644042 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d680a5-5ef1-40fc-b270-b7fd5de4abd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:49:54.709284 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-17d680a5-5ef1-40fc-b270-b7fd5de4abd6 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:49:54.710458 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-17d680a5-5ef1-40fc-b270-b7fd5de4abd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:49:54.804376 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 665/1324] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:49:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:49:54.844827 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd504935-90c5-464a-a13c-9794560e415c None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:49:54.846435 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-cd504935-90c5-464a-a13c-9794560e415c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:49:54.950577 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd504935-90c5-464a-a13c-9794560e415c None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:49:55.022655 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8df832-d0e6-41fd-9297-a4936ecae668 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:49:55.023336 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8df832-d0e6-41fd-9297-a4936ecae668 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:49:55.023622 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8df832-d0e6-41fd-9297-a4936ecae668 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:49:55.023967 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8df832-d0e6-41fd-9297-a4936ecae668 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:49:55.026168 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-cb8df832-d0e6-41fd-9297-a4936ecae668 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:49:55.137142 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8df832-d0e6-41fd-9297-a4936ecae668 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:49:55.138573 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8df832-d0e6-41fd-9297-a4936ecae668 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:49:55.157488 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 660/1325] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:49:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:49:57.530629 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6475ee-7070-4bed-b037-8348cafd2552 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:49:57.531206 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6475ee-7070-4bed-b037-8348cafd2552 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:49:57.531291 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6475ee-7070-4bed-b037-8348cafd2552 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:49:57.532785 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 666/1326] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:49:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:49:57.538235 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f02d2d-25bd-47d6-8140-305177bc56b9 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:49:57.538915 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f02d2d-25bd-47d6-8140-305177bc56b9 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:49:57.538915 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f02d2d-25bd-47d6-8140-305177bc56b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:49:57.608245 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-61f02d2d-25bd-47d6-8140-305177bc56b9 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:49:57.609487 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-61f02d2d-25bd-47d6-8140-305177bc56b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:49:57.708291 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 661/1327] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:49:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 14:49:57.743934 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6475ee-7070-4bed-b037-8348cafd2552 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:49:57.745583 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-ec6475ee-7070-4bed-b037-8348cafd2552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:49:57.856757 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6475ee-7070-4bed-b037-8348cafd2552 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:49:57.932372 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36a8706-0cc5-467b-a436-dbfe3acc1188 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:49:57.933092 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a8706-0cc5-467b-a436-dbfe3acc1188 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:49:57.933423 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a8706-0cc5-467b-a436-dbfe3acc1188 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:49:57.933750 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a8706-0cc5-467b-a436-dbfe3acc1188 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:49:57.935591 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-b36a8706-0cc5-467b-a436-dbfe3acc1188 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:49:58.044212 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36a8706-0cc5-467b-a436-dbfe3acc1188 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:49:58.045673 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36a8706-0cc5-467b-a436-dbfe3acc1188 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:49:58.063282 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 667/1328] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:49:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:50:00.405398 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4afa969-4217-42d5-a9fd-e79ec8046d63 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:00.406077 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4afa969-4217-42d5-a9fd-e79ec8046d63 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:00.406077 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4afa969-4217-42d5-a9fd-e79ec8046d63 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:00.407302 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 662/1329] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:50:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:50:00.412623 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877a998-5a6b-409a-b3bf-665556e848f4 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:00.413023 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877a998-5a6b-409a-b3bf-665556e848f4 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:00.413335 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877a998-5a6b-409a-b3bf-665556e848f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:00.479954 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-8877a998-5a6b-409a-b3bf-665556e848f4 None None] MFA Rules not processed for user `f26c0bc8ce6d4932bba691c04bec3ded`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:50:00.481216 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-8877a998-5a6b-409a-b3bf-665556e848f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:50:00.575494 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 668/1330] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:50:00 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:50:00.612158 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4afa969-4217-42d5-a9fd-e79ec8046d63 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:50:00.613726 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b4afa969-4217-42d5-a9fd-e79ec8046d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:50:00.716798 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4afa969-4217-42d5-a9fd-e79ec8046d63 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:50:00.791614 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1078ab5-0d25-41c4-bb23-c9c358a5a638 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:50:00.792268 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1078ab5-0d25-41c4-bb23-c9c358a5a638 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:00.792549 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1078ab5-0d25-41c4-bb23-c9c358a5a638 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:00.792860 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1078ab5-0d25-41c4-bb23-c9c358a5a638 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:00.794497 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-e1078ab5-0d25-41c4-bb23-c9c358a5a638 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:50:00.903488 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1078ab5-0d25-41c4-bb23-c9c358a5a638 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:50:00.905098 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1078ab5-0d25-41c4-bb23-c9c358a5a638 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:50:00.924093 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 663/1331] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:50:00 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:50:03.471615 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6585ac1-ea37-4d2d-bde1-3c139bbe4953 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:03.472164 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6585ac1-ea37-4d2d-bde1-3c139bbe4953 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:03.472164 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6585ac1-ea37-4d2d-bde1-3c139bbe4953 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:03.473472 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 669/1332] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:50:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:50:03.479645 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e64caa-578d-4999-8320-84f859de569c None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:03.480160 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e64caa-578d-4999-8320-84f859de569c None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:03.480278 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e64caa-578d-4999-8320-84f859de569c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:03.548522 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-19e64caa-578d-4999-8320-84f859de569c None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:50:03.549761 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-19e64caa-578d-4999-8320-84f859de569c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:50:03.644764 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 664/1333] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:50:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:50:03.682441 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6585ac1-ea37-4d2d-bde1-3c139bbe4953 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:50:03.683939 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-d6585ac1-ea37-4d2d-bde1-3c139bbe4953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:50:03.785820 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6585ac1-ea37-4d2d-bde1-3c139bbe4953 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:50:03.860955 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c478ae-b466-4a0d-9cbe-7c7f6f20f7e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:50:03.861613 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c478ae-b466-4a0d-9cbe-7c7f6f20f7e0 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:03.861895 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c478ae-b466-4a0d-9cbe-7c7f6f20f7e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:03.862170 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c478ae-b466-4a0d-9cbe-7c7f6f20f7e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:03.863840 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f1c478ae-b466-4a0d-9cbe-7c7f6f20f7e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:50:03.972324 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c478ae-b466-4a0d-9cbe-7c7f6f20f7e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:50:03.973702 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c478ae-b466-4a0d-9cbe-7c7f6f20f7e0 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:50:03.991527 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 670/1334] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:50:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:50:06.341301 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5050e459-03d2-464b-81be-5f7207cf84c9 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:06.341709 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5050e459-03d2-464b-81be-5f7207cf84c9 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:06.341941 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5050e459-03d2-464b-81be-5f7207cf84c9 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:06.343528 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 665/1335] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:50:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:50:06.349145 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192cef8-f5c9-4925-bcdd-202ca2309fcf None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:06.349549 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192cef8-f5c9-4925-bcdd-202ca2309fcf None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:06.349788 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192cef8-f5c9-4925-bcdd-202ca2309fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:06.419285 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-9192cef8-f5c9-4925-bcdd-202ca2309fcf None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:50:06.420417 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-9192cef8-f5c9-4925-bcdd-202ca2309fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:50:06.519695 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 671/1336] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:50:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:50:06.561084 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5050e459-03d2-464b-81be-5f7207cf84c9 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:50:06.562612 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-5050e459-03d2-464b-81be-5f7207cf84c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:50:06.668009 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5050e459-03d2-464b-81be-5f7207cf84c9 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:50:06.738853 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a8cad-b3fe-4944-a529-f7507a5330a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:50:06.739504 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a8cad-b3fe-4944-a529-f7507a5330a0 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:06.739783 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a8cad-b3fe-4944-a529-f7507a5330a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:06.740059 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a8cad-b3fe-4944-a529-f7507a5330a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:06.741777 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-637a8cad-b3fe-4944-a529-f7507a5330a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:50:06.846878 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a8cad-b3fe-4944-a529-f7507a5330a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:50:06.848302 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a8cad-b3fe-4944-a529-f7507a5330a0 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:50:06.867062 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 666/1337] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:50:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:50:09.467505 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c2fca-b5c1-45b2-9aa5-fb986a5f96c5 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:09.468052 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c2fca-b5c1-45b2-9aa5-fb986a5f96c5 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:09.468052 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c2fca-b5c1-45b2-9aa5-fb986a5f96c5 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:09.469431 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 672/1338] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:50:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:50:09.474736 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048704f6-1f92-48d9-acbc-051a0b481077 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:09.475290 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048704f6-1f92-48d9-acbc-051a0b481077 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:09.475290 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048704f6-1f92-48d9-acbc-051a0b481077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:09.539972 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-048704f6-1f92-48d9-acbc-051a0b481077 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:50:09.540984 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-048704f6-1f92-48d9-acbc-051a0b481077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:50:09.646977 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 667/1339] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:50:09 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 14:50:09.684801 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c2fca-b5c1-45b2-9aa5-fb986a5f96c5 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:50:09.686340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-714c2fca-b5c1-45b2-9aa5-fb986a5f96c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:50:09.786426 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c2fca-b5c1-45b2-9aa5-fb986a5f96c5 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:50:09.858772 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eb32d3-5a46-41b6-b14c-434a5dcc0e7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:50:09.859425 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eb32d3-5a46-41b6-b14c-434a5dcc0e7b service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:09.859707 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eb32d3-5a46-41b6-b14c-434a5dcc0e7b service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:09.859984 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eb32d3-5a46-41b6-b14c-434a5dcc0e7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:09.861724 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-11eb32d3-5a46-41b6-b14c-434a5dcc0e7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:50:09.969836 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eb32d3-5a46-41b6-b14c-434a5dcc0e7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:50:09.971289 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eb32d3-5a46-41b6-b14c-434a5dcc0e7b service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:50:09.988802 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 673/1340] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:50:09 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:50:14.313173 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89f1c6-53b1-481e-b7a8-1ab60fd14f55 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:14.313737 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89f1c6-53b1-481e-b7a8-1ab60fd14f55 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:14.313737 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89f1c6-53b1-481e-b7a8-1ab60fd14f55 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:14.315075 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 668/1341] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:50:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:50:14.320321 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1e898-15c8-4acf-826f-2a9ba3b738fc None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:14.321288 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1e898-15c8-4acf-826f-2a9ba3b738fc None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:14.321532 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1e898-15c8-4acf-826f-2a9ba3b738fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:14.393180 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-eec1e898-15c8-4acf-826f-2a9ba3b738fc None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:50:14.394299 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-eec1e898-15c8-4acf-826f-2a9ba3b738fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:50:14.498137 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 674/1342] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:50:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:50:14.537551 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae89f1c6-53b1-481e-b7a8-1ab60fd14f55 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:50:14.539947 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ae89f1c6-53b1-481e-b7a8-1ab60fd14f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:50:14.637732 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae89f1c6-53b1-481e-b7a8-1ab60fd14f55 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:50:14.641240 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcd45a9-74aa-4f0d-95ef-a965b1de56e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:50:14.641882 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd45a9-74aa-4f0d-95ef-a965b1de56e8 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:14.642160 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd45a9-74aa-4f0d-95ef-a965b1de56e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:14.642433 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd45a9-74aa-4f0d-95ef-a965b1de56e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:14.651635 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcd45a9-74aa-4f0d-95ef-a965b1de56e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:50:14.653375 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcd45a9-74aa-4f0d-95ef-a965b1de56e8 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:50:14.663778 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 669/1343] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:50:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:50:17.571471 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1110c27-e072-4fa7-9d5a-10fef03d8825 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:17.572096 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1110c27-e072-4fa7-9d5a-10fef03d8825 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:17.572096 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1110c27-e072-4fa7-9d5a-10fef03d8825 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:17.576678 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 675/1344] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:50:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:50:17.579166 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f906e-8ee3-4e4b-90f5-48e98ae7dc05 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:17.579673 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f906e-8ee3-4e4b-90f5-48e98ae7dc05 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:17.579673 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f906e-8ee3-4e4b-90f5-48e98ae7dc05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:17.609267 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-743f906e-8ee3-4e4b-90f5-48e98ae7dc05 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:50:17.610448 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-743f906e-8ee3-4e4b-90f5-48e98ae7dc05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:50:17.688525 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 670/1345] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:50:17 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:50:17.725263 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1110c27-e072-4fa7-9d5a-10fef03d8825 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:50:17.736409 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1110c27-e072-4fa7-9d5a-10fef03d8825 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:50:17.740031 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f44936-17fa-406c-a995-00aa15948a8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:50:17.740648 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f44936-17fa-406c-a995-00aa15948a8f service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:17.740936 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f44936-17fa-406c-a995-00aa15948a8f service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:17.741350 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f44936-17fa-406c-a995-00aa15948a8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:17.750547 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f44936-17fa-406c-a995-00aa15948a8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:50:17.751976 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f44936-17fa-406c-a995-00aa15948a8f service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:50:17.761503 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 676/1346] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:50:17 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:50:22.042600 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c093847f-d111-4949-a34f-0e6f105b90ae None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:22.043128 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c093847f-d111-4949-a34f-0e6f105b90ae None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:22.043226 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c093847f-d111-4949-a34f-0e6f105b90ae None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:22.044512 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 671/1347] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:50:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:50:22.049898 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b424e-d5f5-4549-952b-e41d259b1cff None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:22.050294 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b424e-d5f5-4549-952b-e41d259b1cff None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:22.050446 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b424e-d5f5-4549-952b-e41d259b1cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:22.069126 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-d89b424e-d5f5-4549-952b-e41d259b1cff None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:50:22.070286 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-d89b424e-d5f5-4549-952b-e41d259b1cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:50:22.079120 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 677/1348] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:50:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:50:22.117566 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c093847f-d111-4949-a34f-0e6f105b90ae None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:50:22.119367 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-c093847f-d111-4949-a34f-0e6f105b90ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:50:22.132108 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c093847f-d111-4949-a34f-0e6f105b90ae None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:50:22.135637 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1756e352-87c4-4f47-9aad-6220bbc89ac2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:50:22.136273 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756e352-87c4-4f47-9aad-6220bbc89ac2 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:22.136554 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756e352-87c4-4f47-9aad-6220bbc89ac2 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:22.136866 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756e352-87c4-4f47-9aad-6220bbc89ac2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:22.145921 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1756e352-87c4-4f47-9aad-6220bbc89ac2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:50:22.147378 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1756e352-87c4-4f47-9aad-6220bbc89ac2 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:50:22.157721 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 672/1349] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:50:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:50:27.777920 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145e62d-72a0-4e54-b181-79d857c4ee88 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:27.778700 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145e62d-72a0-4e54-b181-79d857c4ee88 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:27.778700 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145e62d-72a0-4e54-b181-79d857c4ee88 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:27.780069 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 678/1350] 10.222.0.23 () {54 vars in 921 bytes} [Fri May 6 14:50:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:50:27.785649 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a0b24-16a2-49dc-91dc-4bf93b699d3b None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:27.786084 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a0b24-16a2-49dc-91dc-4bf93b699d3b None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:27.786205 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a0b24-16a2-49dc-91dc-4bf93b699d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:27.880461 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-fe3a0b24-16a2-49dc-91dc-4bf93b699d3b None None] MFA Rules not processed for user `887a619ef5b5417db98ff15a9f712724`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:50:27.881727 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-fe3a0b24-16a2-49dc-91dc-4bf93b699d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:50:27.956091 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 673/1351] 10.222.0.23 () {58 vars in 1018 bytes} [Fri May 6 14:50:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 14:50:32.500881 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d5d0b-c370-4fa0-8ef9-1584da75e0bd None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:32.501421 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d5d0b-c370-4fa0-8ef9-1584da75e0bd None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:32.501421 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d5d0b-c370-4fa0-8ef9-1584da75e0bd None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:32.503135 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 679/1352] 10.222.0.23 () {54 vars in 921 bytes} [Fri May 6 14:50:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:50:32.508879 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27e74a-7821-49d3-bd58-bcacd2febb7b None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:32.509751 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27e74a-7821-49d3-bd58-bcacd2febb7b None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:32.510275 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27e74a-7821-49d3-bd58-bcacd2febb7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:32.531161 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-7f27e74a-7821-49d3-bd58-bcacd2febb7b None None] MFA Rules not processed for user `887a619ef5b5417db98ff15a9f712724`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:50:32.532919 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-7f27e74a-7821-49d3-bd58-bcacd2febb7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:50:32.543894 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 674/1353] 10.222.0.23 () {58 vars in 1018 bytes} [Fri May 6 14:50:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:50:33.979422 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf190e3-68ed-40cb-9978-6e5f7907cc66 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:33.979982 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf190e3-68ed-40cb-9978-6e5f7907cc66 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:33.980222 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf190e3-68ed-40cb-9978-6e5f7907cc66 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:33.982453 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 680/1354] 10.222.0.23 () {54 vars in 921 bytes} [Fri May 6 14:50:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:50:33.988828 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0188eaf3-3de0-49c9-a587-6525b2d831ab None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:33.989634 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0188eaf3-3de0-49c9-a587-6525b2d831ab None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:33.989912 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0188eaf3-3de0-49c9-a587-6525b2d831ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:34.010728 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-0188eaf3-3de0-49c9-a587-6525b2d831ab None None] MFA Rules not processed for user `887a619ef5b5417db98ff15a9f712724`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:50:34.011980 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-0188eaf3-3de0-49c9-a587-6525b2d831ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:50:34.022472 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 675/1355] 10.222.0.23 () {58 vars in 1018 bytes} [Fri May 6 14:50:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:50:37.942274 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2812cb7-b40d-4388-98ae-8ca69dcd164b None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:37.942786 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2812cb7-b40d-4388-98ae-8ca69dcd164b None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:37.942786 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2812cb7-b40d-4388-98ae-8ca69dcd164b None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:37.944232 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 681/1356] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:50:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:50:37.949636 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d179f0-eda5-4cf8-b3f2-bb20d334d3a7 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:37.950132 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d179f0-eda5-4cf8-b3f2-bb20d334d3a7 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:37.950218 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d179f0-eda5-4cf8-b3f2-bb20d334d3a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:37.969516 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-a4d179f0-eda5-4cf8-b3f2-bb20d334d3a7 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:50:37.970736 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-a4d179f0-eda5-4cf8-b3f2-bb20d334d3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:50:37.981667 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 676/1357] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:50:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:50:38.033514 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2656ecd-d9b4-4756-9dd1-9350f1079f5a None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:38.033985 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2656ecd-d9b4-4756-9dd1-9350f1079f5a None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:38.034397 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2656ecd-d9b4-4756-9dd1-9350f1079f5a None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:38.035941 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 682/1358] 10.222.0.23 () {54 vars in 960 bytes} [Fri May 6 14:50:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:50:38.042029 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2552cc-ad60-4176-9c42-8fa6f5a8bac4 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:38.042325 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2552cc-ad60-4176-9c42-8fa6f5a8bac4 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:38.042761 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2552cc-ad60-4176-9c42-8fa6f5a8bac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:38.094266 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-7a2552cc-ad60-4176-9c42-8fa6f5a8bac4 None None] MFA Rules not processed for user `fc8cf09642ec4e10a80badcf8b585ef8`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:50:38.096223 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-7a2552cc-ad60-4176-9c42-8fa6f5a8bac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:50:38.189921 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 677/1359] 10.222.0.23 () {58 vars in 1057 bytes} [Fri May 6 14:50:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:50:38.199536 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2656ecd-d9b4-4756-9dd1-9350f1079f5a None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:50:38.214294 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2656ecd-d9b4-4756-9dd1-9350f1079f5a None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:50:38.219780 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b719bf2-ce64-4538-a7d2-2763f4032ea5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:50:38.220510 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b719bf2-ce64-4538-a7d2-2763f4032ea5 service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:38.220894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b719bf2-ce64-4538-a7d2-2763f4032ea5 service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:38.221241 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b719bf2-ce64-4538-a7d2-2763f4032ea5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:38.229699 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b719bf2-ce64-4538-a7d2-2763f4032ea5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:50:38.231455 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b719bf2-ce64-4538-a7d2-2763f4032ea5 service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:50:38.241312 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 683/1360] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:50:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 14:50:40.797452 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2119d-21d4-4bca-87bf-4990bdab9a4d None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:40.798040 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2119d-21d4-4bca-87bf-4990bdab9a4d None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:40.798040 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2119d-21d4-4bca-87bf-4990bdab9a4d None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:40.799325 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 678/1361] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:50:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:50:40.804812 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccba26f-805e-4751-b74b-1c27e7a765db None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:40.805720 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccba26f-805e-4751-b74b-1c27e7a765db None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:40.806025 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccba26f-805e-4751-b74b-1c27e7a765db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:40.824743 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-5ccba26f-805e-4751-b74b-1c27e7a765db None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:50:40.825916 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-5ccba26f-805e-4751-b74b-1c27e7a765db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:50:40.833842 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 684/1362] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:50:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:50:40.880664 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2119d-21d4-4bca-87bf-4990bdab9a4d None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:50:40.895960 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2119d-21d4-4bca-87bf-4990bdab9a4d None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:50:40.903039 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377beaad-475e-42da-9181-8d311633f0f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:50:40.903969 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377beaad-475e-42da-9181-8d311633f0f2 service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:40.904278 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377beaad-475e-42da-9181-8d311633f0f2 service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:40.904554 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377beaad-475e-42da-9181-8d311633f0f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:40.918467 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377beaad-475e-42da-9181-8d311633f0f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:50:40.924328 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377beaad-475e-42da-9181-8d311633f0f2 service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:50:40.936836 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 679/1363] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:50:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:50:43.532275 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c250b01-d3bc-4c79-95d5-4e70d4fa7ca5 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:43.532813 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c250b01-d3bc-4c79-95d5-4e70d4fa7ca5 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:43.532921 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c250b01-d3bc-4c79-95d5-4e70d4fa7ca5 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:43.534286 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 685/1364] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:50:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:50:43.540541 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522d57c-af02-4852-92ae-9aa994aed8c1 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:43.541437 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522d57c-af02-4852-92ae-9aa994aed8c1 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:43.541873 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522d57c-af02-4852-92ae-9aa994aed8c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:43.561263 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-e522d57c-af02-4852-92ae-9aa994aed8c1 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:50:43.562461 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-e522d57c-af02-4852-92ae-9aa994aed8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:50:43.571431 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 680/1365] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:50:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:50:43.623285 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c022a133-a54a-4185-adbd-783452e042b6 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:43.623573 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c022a133-a54a-4185-adbd-783452e042b6 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:43.623853 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c022a133-a54a-4185-adbd-783452e042b6 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:43.625258 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 686/1366] 10.222.0.23 () {54 vars in 960 bytes} [Fri May 6 14:50:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:50:43.630724 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de274b36-2a98-4ff1-883e-2d07907ca9bd None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:43.630954 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de274b36-2a98-4ff1-883e-2d07907ca9bd None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:43.631233 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de274b36-2a98-4ff1-883e-2d07907ca9bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:43.649337 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-de274b36-2a98-4ff1-883e-2d07907ca9bd None None] MFA Rules not processed for user `fc8cf09642ec4e10a80badcf8b585ef8`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:50:43.650515 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-de274b36-2a98-4ff1-883e-2d07907ca9bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:50:43.660089 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 681/1367] 10.222.0.23 () {58 vars in 1057 bytes} [Fri May 6 14:50:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:50:43.668420 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c022a133-a54a-4185-adbd-783452e042b6 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:50:43.701639 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c022a133-a54a-4185-adbd-783452e042b6 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:50:43.708553 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909311bf-b9fe-455e-ad64-f6743461ff6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:50:43.709355 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909311bf-b9fe-455e-ad64-f6743461ff6a service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:43.709715 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909311bf-b9fe-455e-ad64-f6743461ff6a service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:43.710151 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909311bf-b9fe-455e-ad64-f6743461ff6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:43.719833 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909311bf-b9fe-455e-ad64-f6743461ff6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:50:43.721635 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909311bf-b9fe-455e-ad64-f6743461ff6a service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:50:43.732641 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 687/1368] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:50:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:50:46.065013 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e86e3-71d2-4030-bf2d-a7822d983f75 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:46.065574 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e86e3-71d2-4030-bf2d-a7822d983f75 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:46.065574 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e86e3-71d2-4030-bf2d-a7822d983f75 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:46.066905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 682/1369] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:50:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:50:46.073293 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f13acb-b745-4fe4-a5cb-f9b22b01a4d5 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:46.073745 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f13acb-b745-4fe4-a5cb-f9b22b01a4d5 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:46.073843 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f13acb-b745-4fe4-a5cb-f9b22b01a4d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:46.092252 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-84f13acb-b745-4fe4-a5cb-f9b22b01a4d5 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:50:46.093550 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-84f13acb-b745-4fe4-a5cb-f9b22b01a4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:50:46.101704 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 688/1370] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:50:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:50:46.146404 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e86e3-71d2-4030-bf2d-a7822d983f75 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:50:46.164595 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e86e3-71d2-4030-bf2d-a7822d983f75 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:50:46.170937 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beabdd0-5cf3-4dcf-938c-bb85ec45f8ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:50:46.171597 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beabdd0-5cf3-4dcf-938c-bb85ec45f8ee service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:46.171883 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beabdd0-5cf3-4dcf-938c-bb85ec45f8ee service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:46.172158 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beabdd0-5cf3-4dcf-938c-bb85ec45f8ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:46.182627 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beabdd0-5cf3-4dcf-938c-bb85ec45f8ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:50:46.184275 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beabdd0-5cf3-4dcf-938c-bb85ec45f8ee service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:50:46.196331 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 683/1371] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:50:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:50:46.379497 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63992eb6-bbc2-496d-99c7-2ac01c6b36c6 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:46.379788 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63992eb6-bbc2-496d-99c7-2ac01c6b36c6 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:46.380265 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63992eb6-bbc2-496d-99c7-2ac01c6b36c6 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:46.381773 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 689/1372] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:50:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:50:46.388540 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f421c3-772e-4c21-811b-087cddaa6011 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:46.388887 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f421c3-772e-4c21-811b-087cddaa6011 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:46.389170 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f421c3-772e-4c21-811b-087cddaa6011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:46.409357 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-a1f421c3-772e-4c21-811b-087cddaa6011 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:50:46.410737 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-a1f421c3-772e-4c21-811b-087cddaa6011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:50:46.420647 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 684/1373] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:50:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:50:46.701185 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05645d96-b97d-44d9-a157-a3324282a9f9 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:46.701479 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05645d96-b97d-44d9-a157-a3324282a9f9 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:46.701750 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05645d96-b97d-44d9-a157-a3324282a9f9 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:46.703108 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 690/1374] 10.222.0.23 () {54 vars in 963 bytes} [Fri May 6 14:50:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:50:46.708137 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea185e-4da3-434e-9236-893d1419cd63 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:46.708595 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea185e-4da3-434e-9236-893d1419cd63 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:46.709043 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea185e-4da3-434e-9236-893d1419cd63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:46.758890 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-c1ea185e-4da3-434e-9236-893d1419cd63 None None] MFA Rules not processed for user `6409b68b4832421ba590d7bc53131f7c`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:50:46.760091 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-c1ea185e-4da3-434e-9236-893d1419cd63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:50:46.823651 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 685/1375] 10.222.0.23 () {58 vars in 1060 bytes} [Fri May 6 14:50:46 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:50:46.833336 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05645d96-b97d-44d9-a157-a3324282a9f9 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:50:46.844942 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05645d96-b97d-44d9-a157-a3324282a9f9 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:50:46.848821 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b834c9-7f1f-4da7-b3d8-27abf6c25b5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:50:46.849679 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b834c9-7f1f-4da7-b3d8-27abf6c25b5f service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:46.850146 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b834c9-7f1f-4da7-b3d8-27abf6c25b5f service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:46.850583 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b834c9-7f1f-4da7-b3d8-27abf6c25b5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:46.861684 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b834c9-7f1f-4da7-b3d8-27abf6c25b5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:50:46.863621 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b834c9-7f1f-4da7-b3d8-27abf6c25b5f service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:50:46.875144 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 691/1376] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:50:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 14:50:48.528455 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470729d-1f78-4fcf-8c09-eb244e03f405 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:48.529342 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470729d-1f78-4fcf-8c09-eb244e03f405 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:48.529342 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470729d-1f78-4fcf-8c09-eb244e03f405 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:48.530738 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 686/1377] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:50:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:50:48.537407 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f5026-a5a1-4b43-9d63-9809e03b1868 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:48.538013 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f5026-a5a1-4b43-9d63-9809e03b1868 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:48.538013 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f5026-a5a1-4b43-9d63-9809e03b1868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:48.557411 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-7c7f5026-a5a1-4b43-9d63-9809e03b1868 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:50:48.558648 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-7c7f5026-a5a1-4b43-9d63-9809e03b1868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:50:48.567873 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 692/1378] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:50:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:50:48.615510 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7470729d-1f78-4fcf-8c09-eb244e03f405 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:50:48.635334 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7470729d-1f78-4fcf-8c09-eb244e03f405 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:50:48.641522 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0e76df-7ca7-4bc1-a50a-1ffd041892a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:50:48.642180 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e76df-7ca7-4bc1-a50a-1ffd041892a4 service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:48.642461 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e76df-7ca7-4bc1-a50a-1ffd041892a4 service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:48.642738 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e76df-7ca7-4bc1-a50a-1ffd041892a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:48.653157 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0e76df-7ca7-4bc1-a50a-1ffd041892a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:50:48.655130 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0e76df-7ca7-4bc1-a50a-1ffd041892a4 service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:50:48.666429 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 687/1379] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:50:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:50:50.971654 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b48f4-1e53-45c0-9218-2a2ac5f023e6 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:50.972211 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b48f4-1e53-45c0-9218-2a2ac5f023e6 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:50.972211 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b48f4-1e53-45c0-9218-2a2ac5f023e6 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:50.973594 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 693/1380] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:50:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:50:50.980034 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85966d3-2c28-4bfd-9bb9-bef76f8403f2 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:50.980490 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85966d3-2c28-4bfd-9bb9-bef76f8403f2 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:50.980883 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85966d3-2c28-4bfd-9bb9-bef76f8403f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:51.003436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-d85966d3-2c28-4bfd-9bb9-bef76f8403f2 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:50:51.004822 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d85966d3-2c28-4bfd-9bb9-bef76f8403f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:50:51.013568 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 688/1381] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:50:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:50:51.062734 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b48f4-1e53-45c0-9218-2a2ac5f023e6 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:50:51.079423 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b48f4-1e53-45c0-9218-2a2ac5f023e6 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:50:51.085120 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca214f-99a5-4f62-b591-9b32f034e61e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:50:51.085882 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca214f-99a5-4f62-b591-9b32f034e61e service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:51.086359 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca214f-99a5-4f62-b591-9b32f034e61e service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:51.086974 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca214f-99a5-4f62-b591-9b32f034e61e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:51.095821 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cca214f-99a5-4f62-b591-9b32f034e61e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:50:51.097553 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cca214f-99a5-4f62-b591-9b32f034e61e service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:50:51.108448 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 694/1382] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:50:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:50:53.393977 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdbbe71-f650-438d-964e-342e9fccaba4 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:53.394505 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdbbe71-f650-438d-964e-342e9fccaba4 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:53.394505 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdbbe71-f650-438d-964e-342e9fccaba4 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:53.395799 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 689/1383] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:50:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:50:53.401928 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e476d74-1180-4083-b560-a59e20f1dbfb None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:53.402421 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e476d74-1180-4083-b560-a59e20f1dbfb None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:53.402916 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e476d74-1180-4083-b560-a59e20f1dbfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:53.422194 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-9e476d74-1180-4083-b560-a59e20f1dbfb None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:50:53.423776 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-9e476d74-1180-4083-b560-a59e20f1dbfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:50:53.436648 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 695/1384] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:50:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:50:53.478412 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdbbe71-f650-438d-964e-342e9fccaba4 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:50:53.493831 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdbbe71-f650-438d-964e-342e9fccaba4 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:50:53.499638 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a42c95-22b1-4381-ae5b-5d7e2aee3ade service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:50:53.500299 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a42c95-22b1-4381-ae5b-5d7e2aee3ade service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:53.500577 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a42c95-22b1-4381-ae5b-5d7e2aee3ade service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:53.500899 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a42c95-22b1-4381-ae5b-5d7e2aee3ade service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:53.510042 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a42c95-22b1-4381-ae5b-5d7e2aee3ade service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:50:53.511629 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a42c95-22b1-4381-ae5b-5d7e2aee3ade service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:50:53.522191 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 690/1385] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:50:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:50:55.771926 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a0496-01a4-4014-9815-44c9d2229ee7 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:55.772657 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a0496-01a4-4014-9815-44c9d2229ee7 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:55.772657 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a0496-01a4-4014-9815-44c9d2229ee7 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:55.774097 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 696/1386] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:50:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:50:55.780182 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8cece-6ba6-44b8-9fe9-327b73961400 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:55.780643 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8cece-6ba6-44b8-9fe9-327b73961400 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:55.780781 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8cece-6ba6-44b8-9fe9-327b73961400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:55.800861 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-6ff8cece-6ba6-44b8-9fe9-327b73961400 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:50:55.802105 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-6ff8cece-6ba6-44b8-9fe9-327b73961400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:50:55.811461 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 691/1387] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:50:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:50:55.860415 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40a0496-01a4-4014-9815-44c9d2229ee7 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:50:55.877852 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40a0496-01a4-4014-9815-44c9d2229ee7 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:50:55.885299 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb30484e-909a-482d-a701-4cea309c703b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:50:55.886101 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30484e-909a-482d-a701-4cea309c703b service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:55.886487 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30484e-909a-482d-a701-4cea309c703b service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:55.886851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30484e-909a-482d-a701-4cea309c703b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:55.897320 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb30484e-909a-482d-a701-4cea309c703b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:50:55.899440 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb30484e-909a-482d-a701-4cea309c703b service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:50:55.912409 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 697/1388] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:50:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:50:58.746971 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12110130-76c8-453a-add4-f32e8a7cff4b None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:58.747537 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12110130-76c8-453a-add4-f32e8a7cff4b None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:58.747665 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12110130-76c8-453a-add4-f32e8a7cff4b None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:58.749471 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 692/1389] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:50:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:50:58.754993 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57516b-3b04-4247-a0c9-5458b13b1fda None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:58.755494 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57516b-3b04-4247-a0c9-5458b13b1fda None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:58.755608 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57516b-3b04-4247-a0c9-5458b13b1fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:58.774595 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-3c57516b-3b04-4247-a0c9-5458b13b1fda None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:50:58.775881 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-3c57516b-3b04-4247-a0c9-5458b13b1fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:50:58.783981 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 698/1390] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:50:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 14:50:58.828500 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12110130-76c8-453a-add4-f32e8a7cff4b None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:50:58.846892 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12110130-76c8-453a-add4-f32e8a7cff4b None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:50:58.855646 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d579961b-6f64-49ac-a888-2b727346da3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:50:58.856353 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579961b-6f64-49ac-a888-2b727346da3c service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:50:58.856651 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579961b-6f64-49ac-a888-2b727346da3c service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:50:58.856951 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579961b-6f64-49ac-a888-2b727346da3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:50:58.865665 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d579961b-6f64-49ac-a888-2b727346da3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:50:58.867971 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d579961b-6f64-49ac-a888-2b727346da3c service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:50:58.878073 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 693/1391] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:50:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:51:01.479449 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba081f-72ca-4ff3-aac4-8d45b4b9385b None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:01.480337 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba081f-72ca-4ff3-aac4-8d45b4b9385b None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:01.480337 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba081f-72ca-4ff3-aac4-8d45b4b9385b None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:01.481404 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 699/1392] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:51:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:51:01.486877 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346b0e0-9a04-4f4f-9f12-a21646402c7c None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:01.487354 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346b0e0-9a04-4f4f-9f12-a21646402c7c None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:01.487462 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346b0e0-9a04-4f4f-9f12-a21646402c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:01.505742 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-e346b0e0-9a04-4f4f-9f12-a21646402c7c None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:51:01.506917 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-e346b0e0-9a04-4f4f-9f12-a21646402c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:51:01.515221 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 694/1393] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:51:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:51:01.560338 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba081f-72ca-4ff3-aac4-8d45b4b9385b None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:51:01.576295 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba081f-72ca-4ff3-aac4-8d45b4b9385b None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:51:01.581765 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cd38c5-8fb9-4641-be63-d1dc8f208e8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:51:01.582403 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd38c5-8fb9-4641-be63-d1dc8f208e8b service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:01.582689 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd38c5-8fb9-4641-be63-d1dc8f208e8b service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:01.582965 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd38c5-8fb9-4641-be63-d1dc8f208e8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:01.591348 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cd38c5-8fb9-4641-be63-d1dc8f208e8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:51:01.592960 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cd38c5-8fb9-4641-be63-d1dc8f208e8b service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:51:01.602947 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 700/1394] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:51:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:51:02.760863 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180d056-9331-4f72-9e84-30b21df38cc4 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:02.761442 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180d056-9331-4f72-9e84-30b21df38cc4 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:02.761442 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180d056-9331-4f72-9e84-30b21df38cc4 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:02.762785 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 695/1395] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:51:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:51:02.768531 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00fedd-bf74-453c-83d5-e9b4962fb8b4 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:02.769151 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00fedd-bf74-453c-83d5-e9b4962fb8b4 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:02.769151 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00fedd-bf74-453c-83d5-e9b4962fb8b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:02.787815 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-5f00fedd-bf74-453c-83d5-e9b4962fb8b4 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:51:02.789832 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-5f00fedd-bf74-453c-83d5-e9b4962fb8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:51:02.797643 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 701/1396] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:51:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:51:02.854108 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84074f82-b3e4-45f5-8779-6e0db6b5fcea None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:02.854394 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84074f82-b3e4-45f5-8779-6e0db6b5fcea None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:02.854672 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84074f82-b3e4-45f5-8779-6e0db6b5fcea None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:02.856029 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 696/1397] 10.222.0.23 () {54 vars in 962 bytes} [Fri May 6 14:51:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:51:02.861416 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3617b-c47e-4f78-b39d-bc56c22c2f1f None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:02.861780 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3617b-c47e-4f78-b39d-bc56c22c2f1f None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:02.862142 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3617b-c47e-4f78-b39d-bc56c22c2f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:02.911084 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-0af3617b-c47e-4f78-b39d-bc56c22c2f1f None None] MFA Rules not processed for user `d9be93f9ff7e46babc8ee87b50b9c5eb`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:51:02.912294 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-0af3617b-c47e-4f78-b39d-bc56c22c2f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:51:02.971912 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 702/1398] 10.222.0.23 () {58 vars in 1059 bytes} [Fri May 6 14:51:02 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:51:02.979192 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84074f82-b3e4-45f5-8779-6e0db6b5fcea None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:51:02.989672 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84074f82-b3e4-45f5-8779-6e0db6b5fcea None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:51:02.993186 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc462081-0356-43d9-a287-3e44faf19249 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9be93f9ff7e46babc8ee87b50b9c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:51:02.993816 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc462081-0356-43d9-a287-3e44faf19249 service glance] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:02.994099 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc462081-0356-43d9-a287-3e44faf19249 service glance] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:02.994374 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc462081-0356-43d9-a287-3e44faf19249 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:03.003086 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc462081-0356-43d9-a287-3e44faf19249 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:51:03.004495 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc462081-0356-43d9-a287-3e44faf19249 service glance] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:51:03.014358 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 697/1399] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:51:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:51:03.437006 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71e1582-f0ea-4cf9-aeb4-d26cd283e4c9 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:03.437372 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71e1582-f0ea-4cf9-aeb4-d26cd283e4c9 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:03.437814 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71e1582-f0ea-4cf9-aeb4-d26cd283e4c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:03.487954 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-f71e1582-f0ea-4cf9-aeb4-d26cd283e4c9 None None] MFA Rules not processed for user `a7dc1bc53c59423e934f7831ea022c73`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:51:03.489285 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f71e1582-f0ea-4cf9-aeb4-d26cd283e4c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:51:03.552587 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 703/1400] 10.222.0.23 () {58 vars in 1009 bytes} [Fri May 6 14:51:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:51:03.600231 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a914c1b8-bf64-4946-a3ac-c80f5dc5a751 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:03.600519 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a914c1b8-bf64-4946-a3ac-c80f5dc5a751 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:03.600829 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a914c1b8-bf64-4946-a3ac-c80f5dc5a751 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:03.602677 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 698/1401] 10.222.0.23 () {54 vars in 942 bytes} [Fri May 6 14:51:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:51:03.609447 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa7669-5bba-4e2d-8f3a-c46f34e3ce94 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:03.609808 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa7669-5bba-4e2d-8f3a-c46f34e3ce94 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:03.610156 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa7669-5bba-4e2d-8f3a-c46f34e3ce94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:03.665460 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-61fa7669-5bba-4e2d-8f3a-c46f34e3ce94 None None] MFA Rules not processed for user `bf9eed62c8f1409282c2af6488426a0f`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:51:03.666750 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-61fa7669-5bba-4e2d-8f3a-c46f34e3ce94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:51:03.746843 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 704/1402] 10.222.0.23 () {58 vars in 1039 bytes} [Fri May 6 14:51:03 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:51:03.757071 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a914c1b8-bf64-4946-a3ac-c80f5dc5a751 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:51:03.770670 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a914c1b8-bf64-4946-a3ac-c80f5dc5a751 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:51:03.776180 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c7d2f-d29f-422f-b8a6-9105b7ed9f9a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf9eed62c8f1409282c2af6488426a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:51:03.776849 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c7d2f-d29f-422f-b8a6-9105b7ed9f9a service swift] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:03.777155 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c7d2f-d29f-422f-b8a6-9105b7ed9f9a service swift] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:03.777399 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c7d2f-d29f-422f-b8a6-9105b7ed9f9a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:03.786049 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750c7d2f-d29f-422f-b8a6-9105b7ed9f9a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:51:03.787628 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750c7d2f-d29f-422f-b8a6-9105b7ed9f9a service swift] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:51:03.797124 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 699/1403] 10.222.0.23 () {60 vars in 1384 bytes} [Fri May 6 14:51:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 06 14:51:03.910591 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb902e2-89a5-4807-9d00-1ba85258cf00 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:03.911155 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb902e2-89a5-4807-9d00-1ba85258cf00 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:03.911155 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb902e2-89a5-4807-9d00-1ba85258cf00 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:03.912654 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 705/1404] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:51:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:51:03.918691 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04981fa3-2b4b-4d45-9582-d892323edf47 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:03.918977 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04981fa3-2b4b-4d45-9582-d892323edf47 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:03.919257 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04981fa3-2b4b-4d45-9582-d892323edf47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:03.939872 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-04981fa3-2b4b-4d45-9582-d892323edf47 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:51:03.941111 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-04981fa3-2b4b-4d45-9582-d892323edf47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:51:03.951968 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 700/1405] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:51:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:51:03.996882 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb902e2-89a5-4807-9d00-1ba85258cf00 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:51:04.013462 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb902e2-89a5-4807-9d00-1ba85258cf00 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:51:04.019346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831a9f25-b279-4c5a-8ec5-0f1a7eb88e8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:51:04.020090 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a9f25-b279-4c5a-8ec5-0f1a7eb88e8d service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:04.020376 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a9f25-b279-4c5a-8ec5-0f1a7eb88e8d service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:04.020856 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a9f25-b279-4c5a-8ec5-0f1a7eb88e8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:04.029815 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831a9f25-b279-4c5a-8ec5-0f1a7eb88e8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:51:04.031386 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831a9f25-b279-4c5a-8ec5-0f1a7eb88e8d service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:51:04.041795 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 706/1406] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:51:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:51:06.390281 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83b712-fec6-4bec-98dd-73b272f65246 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:06.391088 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83b712-fec6-4bec-98dd-73b272f65246 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:06.391389 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83b712-fec6-4bec-98dd-73b272f65246 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:06.392767 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 701/1407] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:51:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:51:06.398682 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b0a957-8dd0-4520-bd47-fa8cea90cabc None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:06.399213 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b0a957-8dd0-4520-bd47-fa8cea90cabc None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:06.399299 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b0a957-8dd0-4520-bd47-fa8cea90cabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:06.421124 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-91b0a957-8dd0-4520-bd47-fa8cea90cabc None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:51:06.422361 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-91b0a957-8dd0-4520-bd47-fa8cea90cabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:51:06.430782 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 707/1408] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:51:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 14:51:06.475337 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f83b712-fec6-4bec-98dd-73b272f65246 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:51:06.494848 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f83b712-fec6-4bec-98dd-73b272f65246 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:51:06.500334 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d6e808-daaf-4d20-96b1-84bbd4f27975 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:51:06.501051 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d6e808-daaf-4d20-96b1-84bbd4f27975 service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:06.501340 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d6e808-daaf-4d20-96b1-84bbd4f27975 service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:06.501620 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d6e808-daaf-4d20-96b1-84bbd4f27975 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:06.510332 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d6e808-daaf-4d20-96b1-84bbd4f27975 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:51:06.511920 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d6e808-daaf-4d20-96b1-84bbd4f27975 service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:51:06.525175 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 702/1409] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:51:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:51:27.823728 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee199e-a39d-477e-ba40-64b7d877c3a6 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:27.824385 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee199e-a39d-477e-ba40-64b7d877c3a6 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:27.824385 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee199e-a39d-477e-ba40-64b7d877c3a6 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:27.825671 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 708/1410] 10.222.0.23 () {54 vars in 918 bytes} [Fri May 6 14:51:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:51:27.831199 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6caa5-e3fe-4dd6-9903-48c200e0c400 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:27.831691 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6caa5-e3fe-4dd6-9903-48c200e0c400 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:27.831812 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6caa5-e3fe-4dd6-9903-48c200e0c400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:27.853189 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-4bb6caa5-e3fe-4dd6-9903-48c200e0c400 None None] MFA Rules not processed for user `887a619ef5b5417db98ff15a9f712724`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:51:27.854387 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-4bb6caa5-e3fe-4dd6-9903-48c200e0c400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:51:27.862824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 703/1411] 10.222.0.23 () {58 vars in 1015 bytes} [Fri May 6 14:51:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 14:51:27.877344 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47672a2e-5c94-40ea-8b43-d429eefaeeda None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:27.877627 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47672a2e-5c94-40ea-8b43-d429eefaeeda None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:27.877905 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47672a2e-5c94-40ea-8b43-d429eefaeeda None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:27.879516 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 709/1412] 10.222.0.23 () {54 vars in 918 bytes} [Fri May 6 14:51:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:51:27.885737 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d11f03-c252-4ce0-8d7b-e415c53acb0c None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:27.886230 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d11f03-c252-4ce0-8d7b-e415c53acb0c None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:27.886649 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d11f03-c252-4ce0-8d7b-e415c53acb0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:27.910588 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-73d11f03-c252-4ce0-8d7b-e415c53acb0c None None] MFA Rules not processed for user `fc8cf09642ec4e10a80badcf8b585ef8`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:51:27.911778 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-73d11f03-c252-4ce0-8d7b-e415c53acb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:51:27.920872 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 704/1413] 10.222.0.23 () {58 vars in 1015 bytes} [Fri May 6 14:51:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:51:30.638743 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56646ef-18c7-462b-9c2f-6a60038f0435 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:30.639385 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56646ef-18c7-462b-9c2f-6a60038f0435 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:30.639385 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56646ef-18c7-462b-9c2f-6a60038f0435 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:30.640665 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 710/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 6 14:51:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:51:30.646296 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e71d63-c341-4510-b2cb-b6cfda6afaeb None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:30.646846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e71d63-c341-4510-b2cb-b6cfda6afaeb None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:30.646846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e71d63-c341-4510-b2cb-b6cfda6afaeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:30.664774 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-82e71d63-c341-4510-b2cb-b6cfda6afaeb None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:51:30.665948 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-82e71d63-c341-4510-b2cb-b6cfda6afaeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:51:30.673476 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 705/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 6 14:51:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:51:30.718777 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56646ef-18c7-462b-9c2f-6a60038f0435 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:51:30.733735 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56646ef-18c7-462b-9c2f-6a60038f0435 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:51:30.739012 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a69339-10b0-47b1-a5b7-68f4d7da38cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:51:30.739639 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a69339-10b0-47b1-a5b7-68f4d7da38cb service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:30.739918 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a69339-10b0-47b1-a5b7-68f4d7da38cb service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:30.740188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a69339-10b0-47b1-a5b7-68f4d7da38cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:30.749420 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a69339-10b0-47b1-a5b7-68f4d7da38cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:51:30.751045 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a69339-10b0-47b1-a5b7-68f4d7da38cb service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:51:30.761063 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 711/1416] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:51:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:51:31.768764 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1c9d1-f819-4849-82f4-cf3f4f2844fa None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:31.769454 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1c9d1-f819-4849-82f4-cf3f4f2844fa None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:31.769454 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1c9d1-f819-4849-82f4-cf3f4f2844fa None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:31.770680 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 706/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 6 14:51:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:51:31.776362 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05797d9-ebb7-4455-a9b4-9bd6d33c45ca None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:31.776866 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05797d9-ebb7-4455-a9b4-9bd6d33c45ca None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:31.777005 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05797d9-ebb7-4455-a9b4-9bd6d33c45ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:31.794998 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-c05797d9-ebb7-4455-a9b4-9bd6d33c45ca None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:51:31.796193 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-c05797d9-ebb7-4455-a9b4-9bd6d33c45ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:51:31.804230 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 712/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 6 14:51:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 14:51:31.961787 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f1c9d1-f819-4849-82f4-cf3f4f2844fa None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:51:31.976692 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f1c9d1-f819-4849-82f4-cf3f4f2844fa None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:51:31.982220 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46006c12-5265-4892-a1e6-27923d09edee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:51:31.982859 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46006c12-5265-4892-a1e6-27923d09edee service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:31.983139 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46006c12-5265-4892-a1e6-27923d09edee service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:31.983415 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46006c12-5265-4892-a1e6-27923d09edee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:31.992164 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46006c12-5265-4892-a1e6-27923d09edee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:51:31.993871 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46006c12-5265-4892-a1e6-27923d09edee service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:51:32.003238 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 707/1419] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:51:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:51:32.971751 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd7427-77d4-4fc6-bb7c-d383b92a3d90 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:32.972328 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd7427-77d4-4fc6-bb7c-d383b92a3d90 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:32.972328 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd7427-77d4-4fc6-bb7c-d383b92a3d90 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:32.973702 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 713/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 6 14:51:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:51:32.979577 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d693568-5056-48b0-87bd-1433dd87d876 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:32.980082 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d693568-5056-48b0-87bd-1433dd87d876 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:32.980218 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d693568-5056-48b0-87bd-1433dd87d876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:32.998580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-5d693568-5056-48b0-87bd-1433dd87d876 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:51:32.999759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-5d693568-5056-48b0-87bd-1433dd87d876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:51:33.007610 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 708/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 6 14:51:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:51:33.167040 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd7427-77d4-4fc6-bb7c-d383b92a3d90 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:51:33.181518 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd7427-77d4-4fc6-bb7c-d383b92a3d90 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:51:33.187074 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef38d264-49dd-4d6a-bced-2d0271aa7f7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:51:33.187718 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38d264-49dd-4d6a-bced-2d0271aa7f7f service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:33.188004 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38d264-49dd-4d6a-bced-2d0271aa7f7f service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:33.188278 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38d264-49dd-4d6a-bced-2d0271aa7f7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:33.198070 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef38d264-49dd-4d6a-bced-2d0271aa7f7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:51:33.199647 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef38d264-49dd-4d6a-bced-2d0271aa7f7f service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:51:33.209054 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 714/1422] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:51:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:51:34.145922 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c81d59-776b-40d7-aafd-4038e8e8ee34 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:34.146511 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c81d59-776b-40d7-aafd-4038e8e8ee34 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:34.146511 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c81d59-776b-40d7-aafd-4038e8e8ee34 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:34.148067 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 709/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 6 14:51:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:51:34.153918 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040dcd3-510e-4f19-845e-7c663f597d2d None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:34.154402 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040dcd3-510e-4f19-845e-7c663f597d2d None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:34.154977 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040dcd3-510e-4f19-845e-7c663f597d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:34.173944 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-e040dcd3-510e-4f19-845e-7c663f597d2d None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:51:34.175141 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e040dcd3-510e-4f19-845e-7c663f597d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:51:34.183640 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 715/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 6 14:51:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:51:34.336627 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c81d59-776b-40d7-aafd-4038e8e8ee34 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:51:34.350613 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c81d59-776b-40d7-aafd-4038e8e8ee34 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:51:34.356130 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2711f77-2030-4eef-8e9e-ddf5a50f8ca5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:51:34.356800 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2711f77-2030-4eef-8e9e-ddf5a50f8ca5 service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:34.357089 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2711f77-2030-4eef-8e9e-ddf5a50f8ca5 service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:34.357362 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2711f77-2030-4eef-8e9e-ddf5a50f8ca5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:34.365593 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2711f77-2030-4eef-8e9e-ddf5a50f8ca5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:51:34.367890 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2711f77-2030-4eef-8e9e-ddf5a50f8ca5 service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:51:34.377279 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 710/1425] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:51:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:51:35.323978 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b4416-54a9-4848-9b9e-d99cb5c33a5d None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:35.324569 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b4416-54a9-4848-9b9e-d99cb5c33a5d None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:35.324569 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b4416-54a9-4848-9b9e-d99cb5c33a5d None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:35.325952 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 716/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 6 14:51:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:51:35.331189 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f3bf6a-f61c-401f-9210-30197a096353 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:35.331627 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f3bf6a-f61c-401f-9210-30197a096353 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:35.331737 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f3bf6a-f61c-401f-9210-30197a096353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:35.350162 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-45f3bf6a-f61c-401f-9210-30197a096353 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:51:35.351433 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-45f3bf6a-f61c-401f-9210-30197a096353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:51:35.363030 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 711/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 6 14:51:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:51:35.525249 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b4416-54a9-4848-9b9e-d99cb5c33a5d None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:51:35.539848 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b4416-54a9-4848-9b9e-d99cb5c33a5d None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:51:35.545275 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a48b9ef-6b93-456a-8570-f57b40ef5a58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:51:35.545909 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48b9ef-6b93-456a-8570-f57b40ef5a58 service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:35.546187 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48b9ef-6b93-456a-8570-f57b40ef5a58 service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:35.546460 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48b9ef-6b93-456a-8570-f57b40ef5a58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:35.555061 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a48b9ef-6b93-456a-8570-f57b40ef5a58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:51:35.556816 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a48b9ef-6b93-456a-8570-f57b40ef5a58 service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:51:35.570150 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 717/1428] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:51:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:51:36.456394 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7421db94-1e4b-4662-9377-8ef6ee2e7ac0 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:36.457084 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7421db94-1e4b-4662-9377-8ef6ee2e7ac0 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:36.457084 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7421db94-1e4b-4662-9377-8ef6ee2e7ac0 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:36.458290 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 712/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 6 14:51:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:51:36.464033 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986da50e-92b0-482c-a709-54b3cc4a7c0e None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:36.464467 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986da50e-92b0-482c-a709-54b3cc4a7c0e None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:36.464752 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986da50e-92b0-482c-a709-54b3cc4a7c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:36.483521 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-986da50e-92b0-482c-a709-54b3cc4a7c0e None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:51:36.484737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-986da50e-92b0-482c-a709-54b3cc4a7c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:51:36.493255 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 718/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 6 14:51:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:51:36.646399 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7421db94-1e4b-4662-9377-8ef6ee2e7ac0 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:51:36.661386 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7421db94-1e4b-4662-9377-8ef6ee2e7ac0 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:51:36.667006 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6f2ed3-7db4-4f6a-be77-42f0bec48168 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:51:36.667646 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f2ed3-7db4-4f6a-be77-42f0bec48168 service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:36.667929 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f2ed3-7db4-4f6a-be77-42f0bec48168 service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:36.668213 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f2ed3-7db4-4f6a-be77-42f0bec48168 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:36.676694 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6f2ed3-7db4-4f6a-be77-42f0bec48168 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:51:36.678199 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6f2ed3-7db4-4f6a-be77-42f0bec48168 service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:51:36.688518 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 713/1431] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:51:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:51:37.588956 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd22e49c-72e1-407a-a615-22e5df21a53d None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:37.589508 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd22e49c-72e1-407a-a615-22e5df21a53d None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:37.589624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd22e49c-72e1-407a-a615-22e5df21a53d None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:37.591170 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 719/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 6 14:51:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:51:37.596545 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663bf4a-2252-4f1d-83d9-fad901f87280 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:37.597106 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663bf4a-2252-4f1d-83d9-fad901f87280 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:37.597217 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663bf4a-2252-4f1d-83d9-fad901f87280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:37.616214 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-c663bf4a-2252-4f1d-83d9-fad901f87280 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:51:37.618792 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-c663bf4a-2252-4f1d-83d9-fad901f87280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:51:37.626753 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 714/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 6 14:51:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:51:37.781834 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd22e49c-72e1-407a-a615-22e5df21a53d None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:51:37.796561 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd22e49c-72e1-407a-a615-22e5df21a53d None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:51:37.802104 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57afcb5b-dc66-4d9c-9fe9-9c5c30cef615 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:51:37.802754 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57afcb5b-dc66-4d9c-9fe9-9c5c30cef615 service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:37.803064 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57afcb5b-dc66-4d9c-9fe9-9c5c30cef615 service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:37.803345 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57afcb5b-dc66-4d9c-9fe9-9c5c30cef615 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:37.812481 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57afcb5b-dc66-4d9c-9fe9-9c5c30cef615 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:51:37.814092 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57afcb5b-dc66-4d9c-9fe9-9c5c30cef615 service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:51:37.824709 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 720/1434] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:51:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:51:38.725282 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669b750-ab3c-4e9e-9b7c-0870b6a04f5d None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:38.725970 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669b750-ab3c-4e9e-9b7c-0870b6a04f5d None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:38.726148 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669b750-ab3c-4e9e-9b7c-0870b6a04f5d None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:38.728205 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 715/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 6 14:51:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:51:38.733708 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b7e59-0f52-4e96-b391-f96152399e9d None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:38.734204 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b7e59-0f52-4e96-b391-f96152399e9d None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:38.734316 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b7e59-0f52-4e96-b391-f96152399e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:38.752721 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-645b7e59-0f52-4e96-b391-f96152399e9d None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:51:38.753928 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-645b7e59-0f52-4e96-b391-f96152399e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:51:38.762867 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 721/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 6 14:51:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:51:38.915362 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9669b750-ab3c-4e9e-9b7c-0870b6a04f5d None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:51:38.930515 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9669b750-ab3c-4e9e-9b7c-0870b6a04f5d None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:51:38.935968 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a684e8-c50a-4d87-af65-c1349bce86b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:51:38.936624 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a684e8-c50a-4d87-af65-c1349bce86b7 service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:38.936916 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a684e8-c50a-4d87-af65-c1349bce86b7 service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:38.937193 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a684e8-c50a-4d87-af65-c1349bce86b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:38.946759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a684e8-c50a-4d87-af65-c1349bce86b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:51:38.948349 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a684e8-c50a-4d87-af65-c1349bce86b7 service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:51:38.958000 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 716/1437] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:51:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:51:39.865086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df663415-61c8-42c5-b881-f2d158d6397a None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:39.865686 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df663415-61c8-42c5-b881-f2d158d6397a None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:39.865686 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df663415-61c8-42c5-b881-f2d158d6397a None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:39.867844 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 722/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 6 14:51:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:51:39.873317 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f4504-8c28-4068-a8c1-9664d56e6909 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:39.873830 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f4504-8c28-4068-a8c1-9664d56e6909 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:39.873966 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f4504-8c28-4068-a8c1-9664d56e6909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:39.892071 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-679f4504-8c28-4068-a8c1-9664d56e6909 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:51:39.893427 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-679f4504-8c28-4068-a8c1-9664d56e6909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:51:39.901459 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 717/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 6 14:51:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 14:51:40.054569 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df663415-61c8-42c5-b881-f2d158d6397a None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:51:40.068848 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df663415-61c8-42c5-b881-f2d158d6397a None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:51:40.075302 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c620c352-27d5-49f9-be3d-d4029c171ac2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:51:40.075979 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c620c352-27d5-49f9-be3d-d4029c171ac2 service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:40.076265 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c620c352-27d5-49f9-be3d-d4029c171ac2 service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:40.076558 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c620c352-27d5-49f9-be3d-d4029c171ac2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:40.085759 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c620c352-27d5-49f9-be3d-d4029c171ac2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:51:40.087903 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c620c352-27d5-49f9-be3d-d4029c171ac2 service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:51:40.097929 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 723/1440] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:51:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:51:42.249642 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4521e-f083-4444-880d-9d6054dee22e None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:42.250450 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4521e-f083-4444-880d-9d6054dee22e None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:42.250652 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4521e-f083-4444-880d-9d6054dee22e None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:42.252238 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 718/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 6 14:51:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:51:42.258282 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448e9fa-2046-4716-b243-58add850f8c5 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:42.258851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448e9fa-2046-4716-b243-58add850f8c5 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:42.258851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448e9fa-2046-4716-b243-58add850f8c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:42.277691 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-3448e9fa-2046-4716-b243-58add850f8c5 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:51:42.278880 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-3448e9fa-2046-4716-b243-58add850f8c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:51:42.286984 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 724/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 6 14:51:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 14:51:42.326087 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4521e-f083-4444-880d-9d6054dee22e None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:51:42.337406 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4521e-f083-4444-880d-9d6054dee22e None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:51:42.341240 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db42ee72-b9a9-480a-b472-60eb276c35ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:51:42.341923 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42ee72-b9a9-480a-b472-60eb276c35ef service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:42.342289 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42ee72-b9a9-480a-b472-60eb276c35ef service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:42.342654 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42ee72-b9a9-480a-b472-60eb276c35ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:42.351918 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db42ee72-b9a9-480a-b472-60eb276c35ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:51:42.353358 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db42ee72-b9a9-480a-b472-60eb276c35ef service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:51:42.362433 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 719/1443] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:51:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 14:51:44.860479 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf16151-620f-4e3a-b529-1b337a4dc68d None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:44.861212 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf16151-620f-4e3a-b529-1b337a4dc68d None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:44.861212 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf16151-620f-4e3a-b529-1b337a4dc68d None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:44.862459 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 725/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 6 14:51:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:51:44.868987 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ec12f-69bb-4b77-a426-28309fc05932 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:44.869463 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ec12f-69bb-4b77-a426-28309fc05932 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:44.869646 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ec12f-69bb-4b77-a426-28309fc05932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:44.887469 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-f77ec12f-69bb-4b77-a426-28309fc05932 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:51:44.888688 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-f77ec12f-69bb-4b77-a426-28309fc05932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:51:44.896393 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 720/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 6 14:51:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:51:44.934739 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf16151-620f-4e3a-b529-1b337a4dc68d None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:51:44.946330 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf16151-620f-4e3a-b529-1b337a4dc68d None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:51:44.949808 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fff1fa3-ac42-4f02-896c-0e8f76c7e9b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:51:44.950437 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff1fa3-ac42-4f02-896c-0e8f76c7e9b2 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:44.950720 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff1fa3-ac42-4f02-896c-0e8f76c7e9b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:44.950992 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff1fa3-ac42-4f02-896c-0e8f76c7e9b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:44.959738 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fff1fa3-ac42-4f02-896c-0e8f76c7e9b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:51:44.961676 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fff1fa3-ac42-4f02-896c-0e8f76c7e9b2 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:51:44.971043 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 726/1446] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:51:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:51:47.613087 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24517fb6-2001-4472-a16f-9645caa984da None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:47.613783 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24517fb6-2001-4472-a16f-9645caa984da None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:47.613783 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24517fb6-2001-4472-a16f-9645caa984da None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:47.615039 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 721/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 6 14:51:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:51:47.621075 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473e640-2c35-4db8-8e33-52497279b527 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:47.621548 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473e640-2c35-4db8-8e33-52497279b527 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:47.621725 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473e640-2c35-4db8-8e33-52497279b527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:47.639874 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-9473e640-2c35-4db8-8e33-52497279b527 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:51:47.641080 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-9473e640-2c35-4db8-8e33-52497279b527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:51:47.649200 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 727/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 6 14:51:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:51:47.689707 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24517fb6-2001-4472-a16f-9645caa984da None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:51:47.701038 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24517fb6-2001-4472-a16f-9645caa984da None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:51:47.704540 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba95646-85bf-4c6a-b76a-7f3cb915228a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:51:47.705203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba95646-85bf-4c6a-b76a-7f3cb915228a service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:47.705486 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba95646-85bf-4c6a-b76a-7f3cb915228a service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:47.705760 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba95646-85bf-4c6a-b76a-7f3cb915228a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:47.714604 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba95646-85bf-4c6a-b76a-7f3cb915228a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:51:47.716345 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba95646-85bf-4c6a-b76a-7f3cb915228a service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:51:47.726068 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 722/1449] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:51:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:51:50.117527 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb89f7-98b2-4cda-8814-af6e82ef7103 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:50.118145 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb89f7-98b2-4cda-8814-af6e82ef7103 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:50.118145 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb89f7-98b2-4cda-8814-af6e82ef7103 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:50.119456 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 728/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 6 14:51:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:51:50.125205 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63540ce5-f2ba-47dd-af9b-1ea7e2935649 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:50.125640 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63540ce5-f2ba-47dd-af9b-1ea7e2935649 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:50.126151 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63540ce5-f2ba-47dd-af9b-1ea7e2935649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:50.144073 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-63540ce5-f2ba-47dd-af9b-1ea7e2935649 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:51:50.145340 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-63540ce5-f2ba-47dd-af9b-1ea7e2935649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:51:50.154277 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 723/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 6 14:51:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:51:50.193302 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bb89f7-98b2-4cda-8814-af6e82ef7103 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:51:50.204763 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bb89f7-98b2-4cda-8814-af6e82ef7103 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:51:50.208276 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e747c-69dd-4526-a80e-778f81f94db1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:51:50.208951 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e747c-69dd-4526-a80e-778f81f94db1 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:50.209238 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e747c-69dd-4526-a80e-778f81f94db1 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:50.209523 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e747c-69dd-4526-a80e-778f81f94db1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:50.218425 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3e747c-69dd-4526-a80e-778f81f94db1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:51:50.219834 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3e747c-69dd-4526-a80e-778f81f94db1 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:51:50.229942 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 729/1452] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:51:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:51:52.539922 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3fddcc-59fa-4848-8e67-a0e4699e0bd5 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:52.540490 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3fddcc-59fa-4848-8e67-a0e4699e0bd5 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:52.540490 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3fddcc-59fa-4848-8e67-a0e4699e0bd5 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:52.541839 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 724/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 6 14:51:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:51:52.547126 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925cd9c8-fc1a-40d0-844a-35f60c97aa91 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:52.547540 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925cd9c8-fc1a-40d0-844a-35f60c97aa91 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:52.547785 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925cd9c8-fc1a-40d0-844a-35f60c97aa91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:52.565794 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-925cd9c8-fc1a-40d0-844a-35f60c97aa91 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:51:52.566999 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-925cd9c8-fc1a-40d0-844a-35f60c97aa91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:51:52.575737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 730/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 6 14:51:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:51:52.613311 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3fddcc-59fa-4848-8e67-a0e4699e0bd5 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:51:52.624441 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3fddcc-59fa-4848-8e67-a0e4699e0bd5 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:51:52.627953 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae719a4-4939-4cff-a341-e5acc37e9f14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:51:52.628594 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae719a4-4939-4cff-a341-e5acc37e9f14 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:52.628912 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae719a4-4939-4cff-a341-e5acc37e9f14 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:52.629189 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae719a4-4939-4cff-a341-e5acc37e9f14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:52.638437 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae719a4-4939-4cff-a341-e5acc37e9f14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:51:52.639852 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae719a4-4939-4cff-a341-e5acc37e9f14 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:51:52.649258 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 725/1455] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:51:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:51:54.958290 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3162a1-d342-4008-819b-6bbccacf1033 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:54.958872 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3162a1-d342-4008-819b-6bbccacf1033 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:54.958872 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3162a1-d342-4008-819b-6bbccacf1033 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:54.960468 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 731/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 6 14:51:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:51:54.965703 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a9f162-da98-4a0b-819c-29b588498859 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:54.966141 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a9f162-da98-4a0b-819c-29b588498859 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:54.966285 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a9f162-da98-4a0b-819c-29b588498859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:54.985299 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-37a9f162-da98-4a0b-819c-29b588498859 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:51:54.986535 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-37a9f162-da98-4a0b-819c-29b588498859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:51:54.994974 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 726/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 6 14:51:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:51:55.031649 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3162a1-d342-4008-819b-6bbccacf1033 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:51:55.042684 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3162a1-d342-4008-819b-6bbccacf1033 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:51:55.046158 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c827a30b-0911-4e41-bc2f-88fe9d3ccaca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:51:55.046791 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c827a30b-0911-4e41-bc2f-88fe9d3ccaca service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:55.047076 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c827a30b-0911-4e41-bc2f-88fe9d3ccaca service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:55.047355 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c827a30b-0911-4e41-bc2f-88fe9d3ccaca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:55.056788 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c827a30b-0911-4e41-bc2f-88fe9d3ccaca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:51:55.058216 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c827a30b-0911-4e41-bc2f-88fe9d3ccaca service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:51:55.067312 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 732/1458] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:51:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:51:57.377457 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a28f22-586c-4f50-8f4a-430f30f2ec93 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:57.378027 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a28f22-586c-4f50-8f4a-430f30f2ec93 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:57.378027 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a28f22-586c-4f50-8f4a-430f30f2ec93 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:57.379329 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 727/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 6 14:51:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:51:57.385150 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf64752-5ddc-4145-8acb-50e07a4fd767 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:57.385575 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf64752-5ddc-4145-8acb-50e07a4fd767 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:57.385720 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf64752-5ddc-4145-8acb-50e07a4fd767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:57.404270 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-bcf64752-5ddc-4145-8acb-50e07a4fd767 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:51:57.405757 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-bcf64752-5ddc-4145-8acb-50e07a4fd767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:51:57.414301 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 733/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 6 14:51:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:51:57.451509 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a28f22-586c-4f50-8f4a-430f30f2ec93 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:51:57.462676 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a28f22-586c-4f50-8f4a-430f30f2ec93 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:51:57.466193 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ec03b4-2679-435b-ad05-7e905a44544c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:51:57.466823 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec03b4-2679-435b-ad05-7e905a44544c service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:57.467104 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec03b4-2679-435b-ad05-7e905a44544c service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:57.467377 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec03b4-2679-435b-ad05-7e905a44544c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:57.476586 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ec03b4-2679-435b-ad05-7e905a44544c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:51:57.478028 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ec03b4-2679-435b-ad05-7e905a44544c service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:51:57.487291 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 728/1461] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:51:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:51:59.800524 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e5116-d44d-4e69-9efa-3a100a9a2946 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:59.801131 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e5116-d44d-4e69-9efa-3a100a9a2946 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:59.801131 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e5116-d44d-4e69-9efa-3a100a9a2946 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:59.802475 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 734/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 6 14:51:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:51:59.808688 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6eafc-d5a4-4e3a-b09c-b86f4a9d2a8c None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:59.809136 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6eafc-d5a4-4e3a-b09c-b86f4a9d2a8c None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:59.809253 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6eafc-d5a4-4e3a-b09c-b86f4a9d2a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:59.826890 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-7cf6eafc-d5a4-4e3a-b09c-b86f4a9d2a8c None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:51:59.828067 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-7cf6eafc-d5a4-4e3a-b09c-b86f4a9d2a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:51:59.835901 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 729/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 6 14:51:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:51:59.872506 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62e5116-d44d-4e69-9efa-3a100a9a2946 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:51:59.883491 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62e5116-d44d-4e69-9efa-3a100a9a2946 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:51:59.886976 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff024f3-73fb-4469-8286-65cae2ab52a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:51:59.887612 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff024f3-73fb-4469-8286-65cae2ab52a9 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:51:59.887897 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff024f3-73fb-4469-8286-65cae2ab52a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:51:59.888176 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff024f3-73fb-4469-8286-65cae2ab52a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:51:59.896910 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff024f3-73fb-4469-8286-65cae2ab52a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:51:59.898498 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff024f3-73fb-4469-8286-65cae2ab52a9 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:51:59.907885 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 735/1464] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:51:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:52:02.142945 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7c2d8-b36f-450c-a81d-4d00fc014842 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:52:02.143490 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7c2d8-b36f-450c-a81d-4d00fc014842 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:52:02.143490 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7c2d8-b36f-450c-a81d-4d00fc014842 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:52:02.145384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 730/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 6 14:52:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:52:02.151835 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eead279-90e0-46da-8c17-40f58f413a6d None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:52:02.152300 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eead279-90e0-46da-8c17-40f58f413a6d None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:52:02.152300 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eead279-90e0-46da-8c17-40f58f413a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:52:02.171080 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-3eead279-90e0-46da-8c17-40f58f413a6d None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:52:02.172253 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-3eead279-90e0-46da-8c17-40f58f413a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:52:02.180393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 736/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 6 14:52:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:52:02.217805 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7c2d8-b36f-450c-a81d-4d00fc014842 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:52:02.228735 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7c2d8-b36f-450c-a81d-4d00fc014842 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:52:02.232148 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ce07c2-32dd-4447-95bd-b1e055b703e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:52:02.232815 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce07c2-32dd-4447-95bd-b1e055b703e9 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:52:02.233093 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce07c2-32dd-4447-95bd-b1e055b703e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:52:02.233363 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce07c2-32dd-4447-95bd-b1e055b703e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:52:02.242344 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ce07c2-32dd-4447-95bd-b1e055b703e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:52:02.243757 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ce07c2-32dd-4447-95bd-b1e055b703e9 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:52:02.252723 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 731/1467] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:52:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:52:04.522997 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fd983-1c86-4324-b5cb-220fc781904d None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:52:04.523569 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fd983-1c86-4324-b5cb-220fc781904d None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:52:04.523569 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fd983-1c86-4324-b5cb-220fc781904d None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:52:04.524930 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 737/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 6 14:52:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:52:04.531183 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b322ef-329b-45e2-b3a4-c5717311f401 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:52:04.531575 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b322ef-329b-45e2-b3a4-c5717311f401 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:52:04.531734 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b322ef-329b-45e2-b3a4-c5717311f401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:52:04.549582 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-82b322ef-329b-45e2-b3a4-c5717311f401 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:52:04.550776 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-82b322ef-329b-45e2-b3a4-c5717311f401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:52:04.559120 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 732/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 6 14:52:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:52:04.596523 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6fd983-1c86-4324-b5cb-220fc781904d None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:52:04.607812 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6fd983-1c86-4324-b5cb-220fc781904d None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:52:04.611298 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d99d602-5f0e-4d65-96dd-21f653556665 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:52:04.611922 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99d602-5f0e-4d65-96dd-21f653556665 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:52:04.612200 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99d602-5f0e-4d65-96dd-21f653556665 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:52:04.612472 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99d602-5f0e-4d65-96dd-21f653556665 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:52:04.622010 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d99d602-5f0e-4d65-96dd-21f653556665 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:52:04.623412 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d99d602-5f0e-4d65-96dd-21f653556665 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:52:04.632746 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 738/1470] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:52:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:52:38.180116 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84862179-4a43-426f-901e-e5352c9ebe31 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:52:38.180673 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84862179-4a43-426f-901e-e5352c9ebe31 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:52:38.180673 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84862179-4a43-426f-901e-e5352c9ebe31 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:52:38.182045 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 733/1471] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:52:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:52:38.187124 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012939b1-a9fc-4870-94a8-97a9e3a1d58f None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:52:38.187549 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012939b1-a9fc-4870-94a8-97a9e3a1d58f None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:52:38.187823 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012939b1-a9fc-4870-94a8-97a9e3a1d58f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:52:38.205992 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-012939b1-a9fc-4870-94a8-97a9e3a1d58f None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:52:38.207159 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-012939b1-a9fc-4870-94a8-97a9e3a1d58f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:52:38.215773 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 739/1472] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:52:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 14:52:38.262810 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35c1a7-fd36-41c2-9121-ba644386b4fa None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:52:38.263097 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35c1a7-fd36-41c2-9121-ba644386b4fa None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:52:38.263372 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35c1a7-fd36-41c2-9121-ba644386b4fa None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:52:38.264738 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 734/1473] 10.222.0.23 () {54 vars in 962 bytes} [Fri May 6 14:52:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:52:38.269556 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35621c51-1ecb-4d71-b6ba-6770a425a489 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:52:38.269841 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35621c51-1ecb-4d71-b6ba-6770a425a489 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:52:38.270110 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35621c51-1ecb-4d71-b6ba-6770a425a489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:52:38.288929 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-35621c51-1ecb-4d71-b6ba-6770a425a489 None None] MFA Rules not processed for user `d9be93f9ff7e46babc8ee87b50b9c5eb`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:52:38.290159 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-35621c51-1ecb-4d71-b6ba-6770a425a489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:52:38.296492 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 740/1474] 10.222.0.23 () {58 vars in 1059 bytes} [Fri May 6 14:52:38 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:52:38.303511 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae35c1a7-fd36-41c2-9121-ba644386b4fa None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:52:38.313863 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae35c1a7-fd36-41c2-9121-ba644386b4fa None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:52:38.317347 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57381753-9f7b-4d10-bb3c-e55ebfb24d99 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9be93f9ff7e46babc8ee87b50b9c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:52:38.317963 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57381753-9f7b-4d10-bb3c-e55ebfb24d99 service glance] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:52:38.318869 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57381753-9f7b-4d10-bb3c-e55ebfb24d99 service glance] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:52:38.319319 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57381753-9f7b-4d10-bb3c-e55ebfb24d99 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:52:38.327910 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57381753-9f7b-4d10-bb3c-e55ebfb24d99 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:52:38.329345 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57381753-9f7b-4d10-bb3c-e55ebfb24d99 service glance] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:52:38.340057 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 735/1475] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:52:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 14:52:40.642846 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8afd7d-c212-4edb-b4f5-3bb05d0728d2 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:52:40.643433 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8afd7d-c212-4edb-b4f5-3bb05d0728d2 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:52:40.643433 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8afd7d-c212-4edb-b4f5-3bb05d0728d2 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:52:40.644801 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 741/1476] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:52:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:52:40.650059 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7740d7e-77e9-4028-9571-1a5035342329 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:52:40.650514 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7740d7e-77e9-4028-9571-1a5035342329 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:52:40.650652 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7740d7e-77e9-4028-9571-1a5035342329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:52:40.668960 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-b7740d7e-77e9-4028-9571-1a5035342329 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:52:40.670147 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b7740d7e-77e9-4028-9571-1a5035342329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:52:40.678386 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 736/1477] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:52:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:52:40.700985 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8afd7d-c212-4edb-b4f5-3bb05d0728d2 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:52:40.713216 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8afd7d-c212-4edb-b4f5-3bb05d0728d2 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:52:40.717971 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a1dc3-2a46-4922-8ff4-2521b4702b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:52:40.718618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a1dc3-2a46-4922-8ff4-2521b4702b8d admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:52:40.718902 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a1dc3-2a46-4922-8ff4-2521b4702b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:52:40.719179 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a1dc3-2a46-4922-8ff4-2521b4702b8d admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:52:40.719901 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053a1dc3-2a46-4922-8ff4-2521b4702b8d admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:52:40.721696 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: WARNING py.warnings [None req-053a1dc3-2a46-4922-8ff4-2521b4702b8d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 06 14:52:40.721696 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: warnings.warn(msg) May 06 14:52:40.721696 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: May 06 14:52:40.722865 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053a1dc3-2a46-4922-8ff4-2521b4702b8d admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:52:40.742807 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 742/1478] 10.222.0.23 () {56 vars in 1108 bytes} [Fri May 6 14:52:40 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:52:43.011258 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d7960-e926-4ef9-93ad-8435ca5f3212 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:52:43.011827 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d7960-e926-4ef9-93ad-8435ca5f3212 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:52:43.011827 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d7960-e926-4ef9-93ad-8435ca5f3212 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:52:43.013288 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 737/1479] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:52:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:52:43.018466 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4e282-c714-470e-af0b-62459c1db5fb None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:52:43.018908 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4e282-c714-470e-af0b-62459c1db5fb None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:52:43.019029 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4e282-c714-470e-af0b-62459c1db5fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:52:43.037461 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-5bc4e282-c714-470e-af0b-62459c1db5fb None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:52:43.038658 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-5bc4e282-c714-470e-af0b-62459c1db5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:52:43.046370 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 743/1480] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:52:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:52:43.089071 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d7960-e926-4ef9-93ad-8435ca5f3212 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:52:43.103175 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d7960-e926-4ef9-93ad-8435ca5f3212 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:52:43.108472 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eaff77-8cd8-4dc5-b04a-55296ac77bd6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:52:43.109141 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eaff77-8cd8-4dc5-b04a-55296ac77bd6 service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:52:43.109424 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eaff77-8cd8-4dc5-b04a-55296ac77bd6 service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:52:43.109699 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eaff77-8cd8-4dc5-b04a-55296ac77bd6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:52:43.118268 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eaff77-8cd8-4dc5-b04a-55296ac77bd6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:52:43.119842 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eaff77-8cd8-4dc5-b04a-55296ac77bd6 service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:52:43.129069 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 738/1481] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:52:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:52:45.562696 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b39776-4b93-401b-aa76-6b7f97a02c9a None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:52:45.563278 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b39776-4b93-401b-aa76-6b7f97a02c9a None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:52:45.563441 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b39776-4b93-401b-aa76-6b7f97a02c9a None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:52:45.565056 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 744/1482] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:52:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:52:45.570720 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58963fab-caf8-4ebd-9f0e-9b30fa218fe2 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:52:45.571174 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58963fab-caf8-4ebd-9f0e-9b30fa218fe2 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:52:45.571427 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58963fab-caf8-4ebd-9f0e-9b30fa218fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:52:45.591154 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-58963fab-caf8-4ebd-9f0e-9b30fa218fe2 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:52:45.592352 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-58963fab-caf8-4ebd-9f0e-9b30fa218fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:52:45.600725 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 739/1483] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:52:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:52:45.635746 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b39776-4b93-401b-aa76-6b7f97a02c9a None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:52:45.646884 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b39776-4b93-401b-aa76-6b7f97a02c9a None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:52:45.650474 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f9ec59-85c4-4bec-9546-f49628890a0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:52:45.651112 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9ec59-85c4-4bec-9546-f49628890a0d service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:52:45.651398 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9ec59-85c4-4bec-9546-f49628890a0d service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:52:45.651672 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9ec59-85c4-4bec-9546-f49628890a0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:52:45.660945 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f9ec59-85c4-4bec-9546-f49628890a0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:52:45.663076 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f9ec59-85c4-4bec-9546-f49628890a0d service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:52:45.672628 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 745/1484] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:52:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:52:47.979203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce45c44-f401-42b5-8626-809e6de1c3a0 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:52:47.979809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce45c44-f401-42b5-8626-809e6de1c3a0 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:52:47.979809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce45c44-f401-42b5-8626-809e6de1c3a0 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:52:47.981187 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 740/1485] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:52:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:52:47.986874 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd89e91-d4f7-49df-9db0-53295a861e0d None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:52:47.987381 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd89e91-d4f7-49df-9db0-53295a861e0d None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:52:47.987519 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd89e91-d4f7-49df-9db0-53295a861e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:52:48.005947 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-1bd89e91-d4f7-49df-9db0-53295a861e0d None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:52:48.007144 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-1bd89e91-d4f7-49df-9db0-53295a861e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:52:48.014863 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 746/1486] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:52:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 14:52:48.053367 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce45c44-f401-42b5-8626-809e6de1c3a0 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:52:48.064592 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce45c44-f401-42b5-8626-809e6de1c3a0 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:52:48.069234 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda38b5f-8b5c-44df-a45c-d1519498190b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:52:48.069867 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda38b5f-8b5c-44df-a45c-d1519498190b service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:52:48.070146 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda38b5f-8b5c-44df-a45c-d1519498190b service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:52:48.070419 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda38b5f-8b5c-44df-a45c-d1519498190b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:52:48.079316 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda38b5f-8b5c-44df-a45c-d1519498190b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:52:48.081339 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda38b5f-8b5c-44df-a45c-d1519498190b service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:52:48.093166 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 741/1487] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:52:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:52:50.491781 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e677f-a307-4c2a-b61d-6accaa1f9701 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:52:50.492435 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e677f-a307-4c2a-b61d-6accaa1f9701 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:52:50.492435 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e677f-a307-4c2a-b61d-6accaa1f9701 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:52:50.493725 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 747/1488] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:52:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:52:50.499040 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60071c29-ce8b-4ab6-941e-d0ce1a8d35c8 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:52:50.499483 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60071c29-ce8b-4ab6-941e-d0ce1a8d35c8 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:52:50.499590 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60071c29-ce8b-4ab6-941e-d0ce1a8d35c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:52:50.517468 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-60071c29-ce8b-4ab6-941e-d0ce1a8d35c8 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:52:50.518643 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-60071c29-ce8b-4ab6-941e-d0ce1a8d35c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:52:50.527284 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 742/1489] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:52:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:52:50.569877 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e677f-a307-4c2a-b61d-6accaa1f9701 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:52:50.580408 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e677f-a307-4c2a-b61d-6accaa1f9701 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:52:50.584541 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b46ccf-19f1-496a-85a9-5b7dac4d8611 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:52:50.585221 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b46ccf-19f1-496a-85a9-5b7dac4d8611 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:52:50.585504 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b46ccf-19f1-496a-85a9-5b7dac4d8611 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:52:50.585776 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b46ccf-19f1-496a-85a9-5b7dac4d8611 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:52:50.594618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b46ccf-19f1-496a-85a9-5b7dac4d8611 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:52:50.596135 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b46ccf-19f1-496a-85a9-5b7dac4d8611 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:52:50.606128 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 748/1490] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:52:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:52:53.147203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc17a9-5bec-4b75-bff2-fa24f34c3870 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:52:53.147787 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc17a9-5bec-4b75-bff2-fa24f34c3870 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:52:53.147787 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc17a9-5bec-4b75-bff2-fa24f34c3870 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:52:53.149202 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 743/1491] 10.222.0.23 () {54 vars in 926 bytes} [Fri May 6 14:52:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:52:53.154707 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961dfee-0b41-4f7e-bbee-c93a7d0f4757 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:52:53.155178 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961dfee-0b41-4f7e-bbee-c93a7d0f4757 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:52:53.155267 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961dfee-0b41-4f7e-bbee-c93a7d0f4757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:52:53.175209 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-d961dfee-0b41-4f7e-bbee-c93a7d0f4757 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:52:53.176533 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-d961dfee-0b41-4f7e-bbee-c93a7d0f4757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:52:53.185634 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 749/1492] 10.222.0.23 () {58 vars in 1023 bytes} [Fri May 6 14:52:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:52:53.225564 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc17a9-5bec-4b75-bff2-fa24f34c3870 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:52:53.237736 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc17a9-5bec-4b75-bff2-fa24f34c3870 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:52:53.242444 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d67f10-f3de-4bb9-9cd4-a46d2c2c17d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:52:53.243173 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d67f10-f3de-4bb9-9cd4-a46d2c2c17d6 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:52:53.243516 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d67f10-f3de-4bb9-9cd4-a46d2c2c17d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:52:53.243862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d67f10-f3de-4bb9-9cd4-a46d2c2c17d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:52:53.253627 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d67f10-f3de-4bb9-9cd4-a46d2c2c17d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:52:53.255077 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d67f10-f3de-4bb9-9cd4-a46d2c2c17d6 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:52:53.266047 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 744/1493] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:52:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:54:21.937810 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada4401-c6fb-4940-94a4-c52bda3473e5 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:54:21.938415 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada4401-c6fb-4940-94a4-c52bda3473e5 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:54:21.938538 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada4401-c6fb-4940-94a4-c52bda3473e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:54:21.957155 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-aada4401-c6fb-4940-94a4-c52bda3473e5 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:54:21.958356 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-aada4401-c6fb-4940-94a4-c52bda3473e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:54:21.966186 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 750/1494] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:54:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:54:21.975697 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd550b3b-475d-4d11-9de2-804e990c577e None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:54:21.976223 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd550b3b-475d-4d11-9de2-804e990c577e None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:54:21.976223 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd550b3b-475d-4d11-9de2-804e990c577e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:54:21.994014 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-bd550b3b-475d-4d11-9de2-804e990c577e None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:54:21.995179 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-bd550b3b-475d-4d11-9de2-804e990c577e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:54:22.003491 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 745/1495] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:54:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:54:22.007988 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aada4401-c6fb-4940-94a4-c52bda3473e5 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:54:22.017712 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aada4401-c6fb-4940-94a4-c52bda3473e5 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:54:22.022978 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df691a1c-1aab-4976-9206-923f6b1fcb51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:54:22.023633 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df691a1c-1aab-4976-9206-923f6b1fcb51 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:54:22.023915 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df691a1c-1aab-4976-9206-923f6b1fcb51 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:54:22.024191 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df691a1c-1aab-4976-9206-923f6b1fcb51 admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:54:22.024937 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df691a1c-1aab-4976-9206-923f6b1fcb51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:54:22.025243 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df691a1c-1aab-4976-9206-923f6b1fcb51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:54:22.026249 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: WARNING py.warnings [None req-df691a1c-1aab-4976-9206-923f6b1fcb51 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 06 14:54:22.026249 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: warnings.warn(msg) May 06 14:54:22.026249 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: May 06 14:54:22.027133 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df691a1c-1aab-4976-9206-923f6b1fcb51 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:54:22.039308 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 751/1496] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 14:54:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:54:22.047324 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd550b3b-475d-4d11-9de2-804e990c577e None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:54:22.056780 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd550b3b-475d-4d11-9de2-804e990c577e None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:54:22.061291 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043fe27d-a381-4d76-af90-223e24b86fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:54:22.061928 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043fe27d-a381-4d76-af90-223e24b86fbb admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:54:22.062208 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043fe27d-a381-4d76-af90-223e24b86fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:54:22.062480 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043fe27d-a381-4d76-af90-223e24b86fbb admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:54:22.063211 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043fe27d-a381-4d76-af90-223e24b86fbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:54:22.065510 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: WARNING py.warnings [None req-043fe27d-a381-4d76-af90-223e24b86fbb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 06 14:54:22.065510 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: warnings.warn(msg) May 06 14:54:22.065510 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: May 06 14:54:22.067273 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043fe27d-a381-4d76-af90-223e24b86fbb admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:54:22.102868 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 746/1497] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:54:22 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 14:54:22.107197 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df691a1c-1aab-4976-9206-923f6b1fcb51 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:54:22.117602 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df691a1c-1aab-4976-9206-923f6b1fcb51 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:54:22.173052 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141e6016-6a33-4668-8b21-4c410c74cde2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:54:22.173730 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141e6016-6a33-4668-8b21-4c410c74cde2 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:54:22.174023 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141e6016-6a33-4668-8b21-4c410c74cde2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:54:22.174305 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141e6016-6a33-4668-8b21-4c410c74cde2 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:54:22.175638 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141e6016-6a33-4668-8b21-4c410c74cde2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:54:22.176672 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: WARNING py.warnings [None req-141e6016-6a33-4668-8b21-4c410c74cde2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 06 14:54:22.176672 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: warnings.warn(msg) May 06 14:54:22.176672 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: May 06 14:54:22.177886 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141e6016-6a33-4668-8b21-4c410c74cde2 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:54:22.198064 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 752/1498] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:54:22 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:54:22.202311 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043fe27d-a381-4d76-af90-223e24b86fbb admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:54:22.212461 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043fe27d-a381-4d76-af90-223e24b86fbb admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:54:22.217160 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc3f5b7-9e41-40c1-9d9e-f4be959478b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:54:22.217824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3f5b7-9e41-40c1-9d9e-f4be959478b4 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:54:22.218109 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3f5b7-9e41-40c1-9d9e-f4be959478b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:54:22.218385 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3f5b7-9e41-40c1-9d9e-f4be959478b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:54:22.219105 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc3f5b7-9e41-40c1-9d9e-f4be959478b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:54:22.220112 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: WARNING py.warnings [None req-7bc3f5b7-9e41-40c1-9d9e-f4be959478b4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 06 14:54:22.220112 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: warnings.warn(msg) May 06 14:54:22.220112 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: May 06 14:54:22.221029 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc3f5b7-9e41-40c1-9d9e-f4be959478b4 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:54:22.233863 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 747/1499] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:54:22.238594 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141e6016-6a33-4668-8b21-4c410c74cde2 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:54:22.248449 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141e6016-6a33-4668-8b21-4c410c74cde2 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:54:22.253851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461a0d57-2d07-4c8c-9d37-95b8ea5230cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:54:22.254500 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a0d57-2d07-4c8c-9d37-95b8ea5230cf admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:54:22.254783 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a0d57-2d07-4c8c-9d37-95b8ea5230cf admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:54:22.255054 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a0d57-2d07-4c8c-9d37-95b8ea5230cf admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:54:22.255759 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461a0d57-2d07-4c8c-9d37-95b8ea5230cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:54:22.256789 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: WARNING py.warnings [None req-461a0d57-2d07-4c8c-9d37-95b8ea5230cf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 06 14:54:22.256789 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: warnings.warn(msg) May 06 14:54:22.256789 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: May 06 14:54:22.257671 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461a0d57-2d07-4c8c-9d37-95b8ea5230cf admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:54:22.271033 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 753/1500] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:54:22.275148 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc3f5b7-9e41-40c1-9d9e-f4be959478b4 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:54:22.285289 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc3f5b7-9e41-40c1-9d9e-f4be959478b4 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:54:22.290337 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9200d32-d641-467c-b644-e57b8f8859ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:54:22.291205 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9200d32-d641-467c-b644-e57b8f8859ac admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:54:22.291483 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9200d32-d641-467c-b644-e57b8f8859ac admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:54:22.291756 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9200d32-d641-467c-b644-e57b8f8859ac admin admin] PATH_INFO: `/v3/projects/f225ce9fa9c3465886830173d2cdd333/users/d523d6b97fd545c997f59b08bc5dcd0b/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:54:22.306376 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9200d32-d641-467c-b644-e57b8f8859ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=f225ce9fa9c3465886830173d2cdd333, user_id=d523d6b97fd545c997f59b08bc5dcd0b, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:54:22.307400 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: WARNING py.warnings [None req-b9200d32-d641-467c-b644-e57b8f8859ac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 06 14:54:22.307400 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: warnings.warn(msg) May 06 14:54:22.307400 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: May 06 14:54:22.309343 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9200d32-d641-467c-b644-e57b8f8859ac admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:54:22.322552 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 748/1501] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:54:22 2022] PUT /identity/v3/projects/f225ce9fa9c3465886830173d2cdd333/users/d523d6b97fd545c997f59b08bc5dcd0b/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:54:22.334205 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6eaf3-d94c-47f3-b288-1b58a7f32d5f None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:54:22.334497 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6eaf3-d94c-47f3-b288-1b58a7f32d5f None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:54:22.334778 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6eaf3-d94c-47f3-b288-1b58a7f32d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:54:22.360341 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-faf6eaf3-d94c-47f3-b288-1b58a7f32d5f None None] MFA Rules not processed for user `d523d6b97fd545c997f59b08bc5dcd0b`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:54:22.361584 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-faf6eaf3-d94c-47f3-b288-1b58a7f32d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:54:22.426240 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 754/1502] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:54:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:54:22.437774 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9200d32-d641-467c-b644-e57b8f8859ac admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:54:22.507356 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9200d32-d641-467c-b644-e57b8f8859ac admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:54:22.512735 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2129401a-f6ba-4460-baf7-c941553cc76c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:54:22.513380 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129401a-f6ba-4460-baf7-c941553cc76c service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:54:22.513658 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129401a-f6ba-4460-baf7-c941553cc76c service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:54:22.513928 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129401a-f6ba-4460-baf7-c941553cc76c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:54:22.522224 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2129401a-f6ba-4460-baf7-c941553cc76c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:54:22.523778 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2129401a-f6ba-4460-baf7-c941553cc76c service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:54:22.532479 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 749/1503] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:54:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:54:22.605203 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095450d3-3670-48b0-8096-8a104402a8be None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:54:22.605498 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095450d3-3670-48b0-8096-8a104402a8be None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:54:22.605780 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095450d3-3670-48b0-8096-8a104402a8be None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:54:22.607056 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 755/1504] 10.222.0.23 () {52 vars in 829 bytes} [Fri May 6 14:54:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:54:22.610565 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2129401a-f6ba-4460-baf7-c941553cc76c service nova] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:54:22.671124 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2129401a-f6ba-4460-baf7-c941553cc76c service nova] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:54:22.676220 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d8f56b-cdbb-4461-ace6-c8b1033e162f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:54:22.677103 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8f56b-cdbb-4461-ace6-c8b1033e162f admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:54:22.677388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8f56b-cdbb-4461-ace6-c8b1033e162f admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:54:22.677665 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8f56b-cdbb-4461-ace6-c8b1033e162f admin admin] PATH_INFO: `/v3/users/d523d6b97fd545c997f59b08bc5dcd0b` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:54:22.678830 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d8f56b-cdbb-4461-ace6-c8b1033e162f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d523d6b97fd545c997f59b08bc5dcd0b)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:54:22.679842 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: WARNING py.warnings [None req-98d8f56b-cdbb-4461-ace6-c8b1033e162f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 06 14:54:22.679842 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: warnings.warn(msg) May 06 14:54:22.679842 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: May 06 14:54:22.681111 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d8f56b-cdbb-4461-ace6-c8b1033e162f admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:54:22.744708 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-98d8f56b-cdbb-4461-ace6-c8b1033e162f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd523d6b97fd545c997f59b08bc5dcd0b'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:54:22.745472 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-98d8f56b-cdbb-4461-ace6-c8b1033e162f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd523d6b97fd545c997f59b08bc5dcd0b'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:54:22.755306 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-98d8f56b-cdbb-4461-ace6-c8b1033e162f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd523d6b97fd545c997f59b08bc5dcd0b'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:54:22.767731 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-98d8f56b-cdbb-4461-ace6-c8b1033e162f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd523d6b97fd545c997f59b08bc5dcd0b'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:54:22.768541 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-98d8f56b-cdbb-4461-ace6-c8b1033e162f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd523d6b97fd545c997f59b08bc5dcd0b'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:54:22.792888 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 750/1505] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 14:54:22 2022] DELETE /identity/v3/users/d523d6b97fd545c997f59b08bc5dcd0b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:54:22.797564 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bbb70f-bf73-44c8-be5a-2ab4e70202fe None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:54:22.797851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bbb70f-bf73-44c8-be5a-2ab4e70202fe None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:54:22.798127 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bbb70f-bf73-44c8-be5a-2ab4e70202fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:54:22.815957 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-d4bbb70f-bf73-44c8-be5a-2ab4e70202fe None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:54:22.817187 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-d4bbb70f-bf73-44c8-be5a-2ab4e70202fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:54:22.874745 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 756/1506] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:54:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:54:22.885160 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d8f56b-cdbb-4461-ace6-c8b1033e162f admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:54:22.886568 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-98d8f56b-cdbb-4461-ace6-c8b1033e162f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:54:22.942548 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d8f56b-cdbb-4461-ace6-c8b1033e162f admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:54:22.946122 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b48e76e-766c-4a83-ae64-e48b98305fa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:54:22.946775 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48e76e-766c-4a83-ae64-e48b98305fa6 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:54:22.947056 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48e76e-766c-4a83-ae64-e48b98305fa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:54:22.947334 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48e76e-766c-4a83-ae64-e48b98305fa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:54:22.956105 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b48e76e-766c-4a83-ae64-e48b98305fa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:54:22.958214 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b48e76e-766c-4a83-ae64-e48b98305fa6 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:54:22.967397 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 751/1507] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:54:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:54:23.332866 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bbb70f-bf73-44c8-be5a-2ab4e70202fe None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:54:23.334527 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-d4bbb70f-bf73-44c8-be5a-2ab4e70202fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:54:23.346358 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bbb70f-bf73-44c8-be5a-2ab4e70202fe None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:54:23.354795 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72ccd6-aed7-419a-b5bc-cbf01a62ba93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:54:23.355757 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72ccd6-aed7-419a-b5bc-cbf01a62ba93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:54:23.356051 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72ccd6-aed7-419a-b5bc-cbf01a62ba93 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:54:23.356323 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72ccd6-aed7-419a-b5bc-cbf01a62ba93 admin admin] PATH_INFO: `/v3/projects/f225ce9fa9c3465886830173d2cdd333` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:54:23.357540 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc72ccd6-aed7-419a-b5bc-cbf01a62ba93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f225ce9fa9c3465886830173d2cdd333)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:54:23.358562 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: WARNING py.warnings [None req-dc72ccd6-aed7-419a-b5bc-cbf01a62ba93 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 06 14:54:23.358562 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: warnings.warn(msg) May 06 14:54:23.358562 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: May 06 14:54:23.359755 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc72ccd6-aed7-419a-b5bc-cbf01a62ba93 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:54:23.428325 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-dc72ccd6-aed7-419a-b5bc-cbf01a62ba93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f225ce9fa9c3465886830173d2cdd333', 'request_id': 'req-dc72ccd6-aed7-419a-b5bc-cbf01a62ba93'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:54:23.434469 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-dc72ccd6-aed7-419a-b5bc-cbf01a62ba93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f225ce9fa9c3465886830173d2cdd333', 'request_id': 'req-dc72ccd6-aed7-419a-b5bc-cbf01a62ba93'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:54:23.445059 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-dc72ccd6-aed7-419a-b5bc-cbf01a62ba93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f225ce9fa9c3465886830173d2cdd333', 'request_id': 'req-dc72ccd6-aed7-419a-b5bc-cbf01a62ba93'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:54:23.464963 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-dc72ccd6-aed7-419a-b5bc-cbf01a62ba93 admin admin] The token cache is being invalidate because project f225ce9fa9c3465886830173d2cdd333 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 14:54:23.465245 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-dc72ccd6-aed7-419a-b5bc-cbf01a62ba93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:54:23.465994 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-dc72ccd6-aed7-419a-b5bc-cbf01a62ba93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:54:23.467533 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 757/1508] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 14:54:23 2022] DELETE /identity/v3/projects/f225ce9fa9c3465886830173d2cdd333 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:56:11.346089 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cb0c34-22f6-420a-a211-661d5ebac31a None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:56:11.346729 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cb0c34-22f6-420a-a211-661d5ebac31a None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:56:11.346729 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cb0c34-22f6-420a-a211-661d5ebac31a None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:56:11.348035 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 752/1509] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 6 14:56:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:56:11.353123 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8a6bd-beef-475f-9d6b-46101e409d82 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:56:11.353586 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8a6bd-beef-475f-9d6b-46101e409d82 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:56:11.353697 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8a6bd-beef-475f-9d6b-46101e409d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:56:11.372194 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-62f8a6bd-beef-475f-9d6b-46101e409d82 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:56:11.373501 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-62f8a6bd-beef-475f-9d6b-46101e409d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:56:11.430802 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 758/1510] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 6 14:56:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 14:56:11.610625 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cb0c34-22f6-420a-a211-661d5ebac31a None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:56:11.612311 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-93cb0c34-22f6-420a-a211-661d5ebac31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:56:11.686952 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cb0c34-22f6-420a-a211-661d5ebac31a None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:56:11.692256 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87107b4-9c53-4ebf-84a9-c48793340daa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:56:11.692936 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87107b4-9c53-4ebf-84a9-c48793340daa service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:56:11.693218 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87107b4-9c53-4ebf-84a9-c48793340daa service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:56:11.693492 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87107b4-9c53-4ebf-84a9-c48793340daa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:56:11.701533 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87107b4-9c53-4ebf-84a9-c48793340daa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:56:11.703610 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87107b4-9c53-4ebf-84a9-c48793340daa service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:56:11.712805 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 753/1511] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:56:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:56:12.671630 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb7410-5b03-46d8-867d-41119e9903fc None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:56:12.672286 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb7410-5b03-46d8-867d-41119e9903fc None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:56:12.672286 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb7410-5b03-46d8-867d-41119e9903fc None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:56:12.673654 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 759/1512] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 6 14:56:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:56:12.678417 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb26f3-6fc4-4087-9747-55b0b709e2b4 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:56:12.678819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb26f3-6fc4-4087-9747-55b0b709e2b4 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:56:12.678968 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb26f3-6fc4-4087-9747-55b0b709e2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:56:12.698173 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-dadb26f3-6fc4-4087-9747-55b0b709e2b4 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:56:12.699365 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-dadb26f3-6fc4-4087-9747-55b0b709e2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:56:12.707750 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 754/1513] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 6 14:56:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:56:12.860417 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cb7410-5b03-46d8-867d-41119e9903fc None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:56:12.874798 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cb7410-5b03-46d8-867d-41119e9903fc None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:56:12.880132 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e02c68-f5b9-4992-a8c7-58cebdbecdcd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:56:12.882540 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e02c68-f5b9-4992-a8c7-58cebdbecdcd service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:56:12.882540 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e02c68-f5b9-4992-a8c7-58cebdbecdcd service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:56:12.882540 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e02c68-f5b9-4992-a8c7-58cebdbecdcd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:56:12.890084 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e02c68-f5b9-4992-a8c7-58cebdbecdcd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:56:12.891672 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e02c68-f5b9-4992-a8c7-58cebdbecdcd service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:56:12.900934 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 760/1514] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:56:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:56:15.149420 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649e1e7-b4cb-47b8-9256-6758ae7bcf84 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:56:15.149959 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649e1e7-b4cb-47b8-9256-6758ae7bcf84 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:56:15.149959 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649e1e7-b4cb-47b8-9256-6758ae7bcf84 None None] PATH_INFO: `/v3` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:56:15.151192 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 755/1515] 10.222.0.26 () {54 vars in 925 bytes} [Fri May 6 14:56:15 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:56:15.160540 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0db9f1-d1f6-43de-b7c6-fa26ecc6aa64 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:56:15.160996 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0db9f1-d1f6-43de-b7c6-fa26ecc6aa64 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:56:15.161142 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0db9f1-d1f6-43de-b7c6-fa26ecc6aa64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:56:15.179427 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-0e0db9f1-d1f6-43de-b7c6-fa26ecc6aa64 None None] MFA Rules not processed for user `887a619ef5b5417db98ff15a9f712724`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:56:15.180665 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-0e0db9f1-d1f6-43de-b7c6-fa26ecc6aa64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:56:15.240753 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 761/1516] 10.222.0.26 () {58 vars in 1015 bytes} [Fri May 6 14:56:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:56:19.965015 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a3a92-f4dd-48df-ab58-44a8bc046ef8 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:56:19.965591 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a3a92-f4dd-48df-ab58-44a8bc046ef8 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:56:19.965702 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a3a92-f4dd-48df-ab58-44a8bc046ef8 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:56:19.967049 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 756/1517] 10.222.0.23 () {54 vars in 960 bytes} [Fri May 6 14:56:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:56:19.972223 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7faa5-284f-4081-8727-09515b12a0c8 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:56:19.972672 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7faa5-284f-4081-8727-09515b12a0c8 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:56:19.972796 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7faa5-284f-4081-8727-09515b12a0c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:56:19.991123 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-3cb7faa5-284f-4081-8727-09515b12a0c8 None None] MFA Rules not processed for user `887a619ef5b5417db98ff15a9f712724`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:56:19.992301 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-3cb7faa5-284f-4081-8727-09515b12a0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:56:20.001313 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 762/1518] 10.222.0.23 () {58 vars in 1057 bytes} [Fri May 6 14:56:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:56:20.010241 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a3a92-f4dd-48df-ab58-44a8bc046ef8 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:56:20.023410 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a3a92-f4dd-48df-ab58-44a8bc046ef8 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:56:20.028249 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1461d33-4af3-4f4e-9af0-99e77938d04b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887a619ef5b5417db98ff15a9f712724', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:56:20.028914 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1461d33-4af3-4f4e-9af0-99e77938d04b service placement] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:56:20.029199 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1461d33-4af3-4f4e-9af0-99e77938d04b service placement] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:56:20.029514 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1461d33-4af3-4f4e-9af0-99e77938d04b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:56:20.037917 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1461d33-4af3-4f4e-9af0-99e77938d04b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:56:20.039486 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1461d33-4af3-4f4e-9af0-99e77938d04b service placement] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:56:20.049921 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 757/1519] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:56:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 14:56:24.258856 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0cf30-a9a4-4b81-a2f5-fb5265ff3896 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:56:24.259420 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0cf30-a9a4-4b81-a2f5-fb5265ff3896 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:56:24.259420 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0cf30-a9a4-4b81-a2f5-fb5265ff3896 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:56:24.260851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 763/1520] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 6 14:56:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:56:24.266233 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477cfff-d844-448f-9d43-6b87e5b814e8 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:56:24.266691 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477cfff-d844-448f-9d43-6b87e5b814e8 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:56:24.266821 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477cfff-d844-448f-9d43-6b87e5b814e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:56:24.285582 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-4477cfff-d844-448f-9d43-6b87e5b814e8 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:56:24.286766 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-4477cfff-d844-448f-9d43-6b87e5b814e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:56:24.294467 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 758/1521] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 6 14:56:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:56:24.447733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c0cf30-a9a4-4b81-a2f5-fb5265ff3896 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:56:24.462754 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c0cf30-a9a4-4b81-a2f5-fb5265ff3896 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:56:24.469177 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fd07ae-91f1-4390-8d79-6a6f7d7125ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:56:24.469813 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd07ae-91f1-4390-8d79-6a6f7d7125ea service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:56:24.470099 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd07ae-91f1-4390-8d79-6a6f7d7125ea service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:56:24.470381 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd07ae-91f1-4390-8d79-6a6f7d7125ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:56:24.478313 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fd07ae-91f1-4390-8d79-6a6f7d7125ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:56:24.479909 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fd07ae-91f1-4390-8d79-6a6f7d7125ea service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:56:24.489606 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 764/1522] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:56:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:56:25.401064 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0bf2b-03f3-4e20-92e5-597220879538 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:56:25.401603 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0bf2b-03f3-4e20-92e5-597220879538 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:56:25.401733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0bf2b-03f3-4e20-92e5-597220879538 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:56:25.403014 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 759/1523] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 6 14:56:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:56:25.408361 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b896974-c2e4-40bc-b061-0e31d5b9ad4c None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:56:25.408857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b896974-c2e4-40bc-b061-0e31d5b9ad4c None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:56:25.408966 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b896974-c2e4-40bc-b061-0e31d5b9ad4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:56:25.427236 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-1b896974-c2e4-40bc-b061-0e31d5b9ad4c None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:56:25.428446 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-1b896974-c2e4-40bc-b061-0e31d5b9ad4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:56:25.437011 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 765/1524] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 6 14:56:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:56:25.590076 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde0bf2b-03f3-4e20-92e5-597220879538 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:56:25.604787 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde0bf2b-03f3-4e20-92e5-597220879538 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:56:25.610828 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d792df-c6c5-4e5e-b38a-174d595b94e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:56:25.611472 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d792df-c6c5-4e5e-b38a-174d595b94e9 service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:56:25.611756 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d792df-c6c5-4e5e-b38a-174d595b94e9 service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:56:25.612036 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d792df-c6c5-4e5e-b38a-174d595b94e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:56:25.620282 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d792df-c6c5-4e5e-b38a-174d595b94e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:56:25.621915 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d792df-c6c5-4e5e-b38a-174d595b94e9 service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:56:25.631188 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 760/1525] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:56:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:56:29.886229 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceafa55-c8a5-4f6b-ad1d-94054e71919d None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:56:29.886780 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceafa55-c8a5-4f6b-ad1d-94054e71919d None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:56:29.886780 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceafa55-c8a5-4f6b-ad1d-94054e71919d None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:56:29.888140 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 766/1526] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 6 14:56:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:56:29.893983 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f44ff-c2ee-43eb-ba24-0b99deb05796 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:56:29.894424 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f44ff-c2ee-43eb-ba24-0b99deb05796 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:56:29.894530 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f44ff-c2ee-43eb-ba24-0b99deb05796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:56:29.912581 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-a99f44ff-c2ee-43eb-ba24-0b99deb05796 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:56:29.913784 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-a99f44ff-c2ee-43eb-ba24-0b99deb05796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:56:29.922269 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 761/1527] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 6 14:56:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:56:29.959330 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceafa55-c8a5-4f6b-ad1d-94054e71919d None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:56:29.961078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-0ceafa55-c8a5-4f6b-ad1d-94054e71919d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:56:30.020124 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceafa55-c8a5-4f6b-ad1d-94054e71919d None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:56:30.023672 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ecd752-250c-4c0b-a038-596d02aa6b76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:56:30.024343 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ecd752-250c-4c0b-a038-596d02aa6b76 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:56:30.024641 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ecd752-250c-4c0b-a038-596d02aa6b76 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:56:30.024918 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ecd752-250c-4c0b-a038-596d02aa6b76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:56:30.034335 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ecd752-250c-4c0b-a038-596d02aa6b76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:56:30.035750 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ecd752-250c-4c0b-a038-596d02aa6b76 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:56:30.045406 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 767/1528] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:56:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:56:32.316062 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61259d5a-0c37-4309-b175-06bbaefd3cef None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:56:32.316667 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61259d5a-0c37-4309-b175-06bbaefd3cef None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:56:32.316667 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61259d5a-0c37-4309-b175-06bbaefd3cef None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:56:32.317980 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 762/1529] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 6 14:56:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:56:32.323266 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d17ac-3ff6-4e2c-a8bb-6762adf486ac None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:56:32.323671 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d17ac-3ff6-4e2c-a8bb-6762adf486ac None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:56:32.323812 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d17ac-3ff6-4e2c-a8bb-6762adf486ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:56:32.342160 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-d38d17ac-3ff6-4e2c-a8bb-6762adf486ac None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:56:32.343426 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-d38d17ac-3ff6-4e2c-a8bb-6762adf486ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:56:32.351157 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 768/1530] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 6 14:56:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:56:32.386891 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61259d5a-0c37-4309-b175-06bbaefd3cef None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:56:32.397637 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61259d5a-0c37-4309-b175-06bbaefd3cef None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:56:32.401107 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c9ee7-8e79-4b29-8d80-46104222bcc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:56:32.401746 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c9ee7-8e79-4b29-8d80-46104222bcc5 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:56:32.402027 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c9ee7-8e79-4b29-8d80-46104222bcc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:56:32.402304 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c9ee7-8e79-4b29-8d80-46104222bcc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:56:32.412073 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479c9ee7-8e79-4b29-8d80-46104222bcc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:56:32.413473 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479c9ee7-8e79-4b29-8d80-46104222bcc5 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:56:32.423288 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 763/1531] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:56:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:56:36.196161 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f5962-d654-4083-8843-a9e57983e182 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:56:36.196784 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f5962-d654-4083-8843-a9e57983e182 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:56:36.196784 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f5962-d654-4083-8843-a9e57983e182 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:56:36.198138 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 769/1532] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 6 14:56:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:56:36.204238 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9eeb8-f6f8-44cd-9aad-425b994d7c46 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:56:36.204706 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9eeb8-f6f8-44cd-9aad-425b994d7c46 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:56:36.204830 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9eeb8-f6f8-44cd-9aad-425b994d7c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:56:36.222740 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-82c9eeb8-f6f8-44cd-9aad-425b994d7c46 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:56:36.223924 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-82c9eeb8-f6f8-44cd-9aad-425b994d7c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:56:36.231663 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 764/1533] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 6 14:56:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:56:36.514008 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f5962-d654-4083-8843-a9e57983e182 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:56:36.515738 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e76f5962-d654-4083-8843-a9e57983e182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:56:36.573455 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f5962-d654-4083-8843-a9e57983e182 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:56:36.577596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cfeee5-bd8f-46c9-8952-3392c4b7b95e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:56:36.578262 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cfeee5-bd8f-46c9-8952-3392c4b7b95e service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:56:36.578548 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cfeee5-bd8f-46c9-8952-3392c4b7b95e service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:56:36.578829 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cfeee5-bd8f-46c9-8952-3392c4b7b95e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:56:36.588287 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cfeee5-bd8f-46c9-8952-3392c4b7b95e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:56:36.589798 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cfeee5-bd8f-46c9-8952-3392c4b7b95e service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:56:36.599393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 770/1534] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:56:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:56:38.937046 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00fa6a-a187-44b3-870f-e081dd9f7732 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:56:38.937658 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00fa6a-a187-44b3-870f-e081dd9f7732 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:56:38.937658 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00fa6a-a187-44b3-870f-e081dd9f7732 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:56:38.938894 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 765/1535] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 6 14:56:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:56:38.944277 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71dcdd3-1342-4159-8872-5f48fccc1639 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:56:38.944754 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71dcdd3-1342-4159-8872-5f48fccc1639 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:56:38.944904 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71dcdd3-1342-4159-8872-5f48fccc1639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:56:38.963081 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-c71dcdd3-1342-4159-8872-5f48fccc1639 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:56:38.964268 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-c71dcdd3-1342-4159-8872-5f48fccc1639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:56:38.972629 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 771/1536] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 6 14:56:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:56:39.256566 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08991a-ab4e-4de8-8932-bcb16a910ce7 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:56:39.256888 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08991a-ab4e-4de8-8932-bcb16a910ce7 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:56:39.257169 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08991a-ab4e-4de8-8932-bcb16a910ce7 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:56:39.258552 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 766/1537] 10.222.0.23 () {54 vars in 963 bytes} [Fri May 6 14:56:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:56:39.264763 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4957bc5-d989-4f6e-b545-dee0216403ba None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:56:39.265055 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4957bc5-d989-4f6e-b545-dee0216403ba None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:56:39.265327 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4957bc5-d989-4f6e-b545-dee0216403ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:56:39.283831 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-f4957bc5-d989-4f6e-b545-dee0216403ba None None] MFA Rules not processed for user `6409b68b4832421ba590d7bc53131f7c`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:56:39.285081 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f4957bc5-d989-4f6e-b545-dee0216403ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:56:39.291600 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 772/1538] 10.222.0.23 () {58 vars in 1060 bytes} [Fri May 6 14:56:39 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:56:39.299499 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08991a-ab4e-4de8-8932-bcb16a910ce7 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:56:39.310684 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08991a-ab4e-4de8-8932-bcb16a910ce7 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:56:39.314541 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e94de4-4afa-4b65-8a81-5ff27846ad4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:56:39.315184 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e94de4-4afa-4b65-8a81-5ff27846ad4d service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:56:39.315467 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e94de4-4afa-4b65-8a81-5ff27846ad4d service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:56:39.315746 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e94de4-4afa-4b65-8a81-5ff27846ad4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:56:39.324581 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e94de4-4afa-4b65-8a81-5ff27846ad4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:56:39.326832 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e94de4-4afa-4b65-8a81-5ff27846ad4d service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:56:39.336300 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 767/1539] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:56:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 14:58:16.645354 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e233fd-88d8-4503-a45a-8762a9bffc95 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:16.646034 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e233fd-88d8-4503-a45a-8762a9bffc95 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:16.646034 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e233fd-88d8-4503-a45a-8762a9bffc95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:16.676926 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-10e233fd-88d8-4503-a45a-8762a9bffc95 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:16.678156 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-10e233fd-88d8-4503-a45a-8762a9bffc95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:16.691307 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 773/1540] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:16.706246 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b0be8-aa12-481d-a4cf-790d1410f1de None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:16.706837 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b0be8-aa12-481d-a4cf-790d1410f1de None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:16.706837 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b0be8-aa12-481d-a4cf-790d1410f1de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:16.733914 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-686b0be8-aa12-481d-a4cf-790d1410f1de None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:16.735185 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-686b0be8-aa12-481d-a4cf-790d1410f1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:16.747891 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 768/1541] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:16.750101 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49beb461-6820-4f1d-b485-bfa7da73027d None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:16.750395 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49beb461-6820-4f1d-b485-bfa7da73027d None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:16.750827 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49beb461-6820-4f1d-b485-bfa7da73027d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:16.761030 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b0be8-aa12-481d-a4cf-790d1410f1de None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:16.771608 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b0be8-aa12-481d-a4cf-790d1410f1de None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:16.779838 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-49beb461-6820-4f1d-b485-bfa7da73027d None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:16.784771 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-49beb461-6820-4f1d-b485-bfa7da73027d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:16.786419 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28757984-9442-4cd2-b0a9-355b0db5b9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:16.787295 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28757984-9442-4cd2-b0a9-355b0db5b9e3 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:16.787582 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28757984-9442-4cd2-b0a9-355b0db5b9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:16.787857 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28757984-9442-4cd2-b0a9-355b0db5b9e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:16.788616 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28757984-9442-4cd2-b0a9-355b0db5b9e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:58:16.788936 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28757984-9442-4cd2-b0a9-355b0db5b9e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:16.789976 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: WARNING py.warnings [None req-28757984-9442-4cd2-b0a9-355b0db5b9e3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 06 14:58:16.789976 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: warnings.warn(msg) May 06 14:58:16.789976 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: May 06 14:58:16.790878 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28757984-9442-4cd2-b0a9-355b0db5b9e3 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:16.812580 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 774/1542] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:16.825538 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38039a2f-dfaf-4d5d-9954-6ea3f6d671b6 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:16.825898 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38039a2f-dfaf-4d5d-9954-6ea3f6d671b6 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:16.826092 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38039a2f-dfaf-4d5d-9954-6ea3f6d671b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:16.832362 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 769/1543] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 14:58:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:58:16.853400 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-38039a2f-dfaf-4d5d-9954-6ea3f6d671b6 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:16.854656 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-38039a2f-dfaf-4d5d-9954-6ea3f6d671b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:16.859505 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28757984-9442-4cd2-b0a9-355b0db5b9e3 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:16.872479 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28757984-9442-4cd2-b0a9-355b0db5b9e3 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:16.882908 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 775/1544] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 14:58:16.883479 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a792d24-3139-4f7e-bbef-e14186cd7bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:16.884157 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a792d24-3139-4f7e-bbef-e14186cd7bfa admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:16.884519 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a792d24-3139-4f7e-bbef-e14186cd7bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:16.884866 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a792d24-3139-4f7e-bbef-e14186cd7bfa admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:16.885549 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a792d24-3139-4f7e-bbef-e14186cd7bfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:16.888133 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a792d24-3139-4f7e-bbef-e14186cd7bfa admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:16.890330 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38039a2f-dfaf-4d5d-9954-6ea3f6d671b6 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:16.903195 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38039a2f-dfaf-4d5d-9954-6ea3f6d671b6 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:16.923062 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d71c5-5a97-4bdc-b044-3ac62ac53189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:16.923765 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d71c5-5a97-4bdc-b044-3ac62ac53189 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:16.924052 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d71c5-5a97-4bdc-b044-3ac62ac53189 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:16.924346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d71c5-5a97-4bdc-b044-3ac62ac53189 admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:16.925119 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d71c5-5a97-4bdc-b044-3ac62ac53189 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:58:16.925428 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d71c5-5a97-4bdc-b044-3ac62ac53189 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:16.927115 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d71c5-5a97-4bdc-b044-3ac62ac53189 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:16.964357 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 770/1545] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:58:16 2022] POST /identity/v3/projects => generated 379 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 14:58:16.966393 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243cdac-9c52-4bd4-9b9b-78e3814ef2ca None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:16.966684 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243cdac-9c52-4bd4-9b9b-78e3814ef2ca None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:16.966974 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243cdac-9c52-4bd4-9b9b-78e3814ef2ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:16.971022 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 776/1546] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 14:58:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:58:16.977418 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d71c5-5a97-4bdc-b044-3ac62ac53189 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:16.998818 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d71c5-5a97-4bdc-b044-3ac62ac53189 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:17.010069 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-4243cdac-9c52-4bd4-9b9b-78e3814ef2ca None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:17.012268 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-4243cdac-9c52-4bd4-9b9b-78e3814ef2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:17.091354 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e4bd5-6581-404a-9726-6ffad642287d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:17.092151 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e4bd5-6581-404a-9726-6ffad642287d admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:17.092434 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e4bd5-6581-404a-9726-6ffad642287d admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:17.092743 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e4bd5-6581-404a-9726-6ffad642287d admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:17.093458 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131e4bd5-6581-404a-9726-6ffad642287d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:17.095452 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131e4bd5-6581-404a-9726-6ffad642287d admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:17.124346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 777/1547] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:58:16 2022] POST /identity/v3/users => generated 290 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:58:17.125305 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e4bd5-6581-404a-9726-6ffad642287d admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:17.137787 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e4bd5-6581-404a-9726-6ffad642287d admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:17.143965 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b750b207-3d5e-4dd3-940f-a524d1de1f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:17.144711 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750b207-3d5e-4dd3-940f-a524d1de1f82 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:17.144995 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750b207-3d5e-4dd3-940f-a524d1de1f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:17.145267 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750b207-3d5e-4dd3-940f-a524d1de1f82 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:17.145966 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b750b207-3d5e-4dd3-940f-a524d1de1f82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:17.147192 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: WARNING py.warnings [None req-b750b207-3d5e-4dd3-940f-a524d1de1f82 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 06 14:58:17.147192 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: warnings.warn(msg) May 06 14:58:17.147192 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: May 06 14:58:17.148550 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b750b207-3d5e-4dd3-940f-a524d1de1f82 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:17.179639 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 778/1548] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:58:17 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:58:17.181045 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48402d2b-7242-40f9-8573-d3a5ea2eaa4f None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:17.181332 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48402d2b-7242-40f9-8573-d3a5ea2eaa4f None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:17.181608 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48402d2b-7242-40f9-8573-d3a5ea2eaa4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:17.202242 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-48402d2b-7242-40f9-8573-d3a5ea2eaa4f None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:17.203449 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-48402d2b-7242-40f9-8573-d3a5ea2eaa4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:17.240897 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 771/1549] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:17.242456 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b73b2a-e153-4e83-94f8-d2f491817b71 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:17.242866 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b73b2a-e153-4e83-94f8-d2f491817b71 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:17.243286 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b73b2a-e153-4e83-94f8-d2f491817b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:17.262840 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-37b73b2a-e153-4e83-94f8-d2f491817b71 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:17.264131 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-37b73b2a-e153-4e83-94f8-d2f491817b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:17.273657 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 779/1550] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:17.274966 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c582a5da-3601-44ae-813e-7e4713342a9b None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:17.275299 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c582a5da-3601-44ae-813e-7e4713342a9b None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:17.275538 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c582a5da-3601-44ae-813e-7e4713342a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:17.301467 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-c582a5da-3601-44ae-813e-7e4713342a9b None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:17.302667 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-c582a5da-3601-44ae-813e-7e4713342a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:17.312045 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 780/1551] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:17.313320 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7459f-1d8a-43f7-a3bb-fe5f729d26a8 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:17.313687 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7459f-1d8a-43f7-a3bb-fe5f729d26a8 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:17.314052 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7459f-1d8a-43f7-a3bb-fe5f729d26a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:17.330801 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 772/1552] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:17.332801 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d42cf-1c50-48ed-87f7-d3414dc32cf7 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:17.332801 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d42cf-1c50-48ed-87f7-d3414dc32cf7 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:17.333122 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d42cf-1c50-48ed-87f7-d3414dc32cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:17.342038 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-a9c7459f-1d8a-43f7-a3bb-fe5f729d26a8 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:17.343246 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-a9c7459f-1d8a-43f7-a3bb-fe5f729d26a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:17.351271 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 781/1553] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:17.352204 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c7459f-1d8a-43f7-a3bb-fe5f729d26a8 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:17.357032 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-a45d42cf-1c50-48ed-87f7-d3414dc32cf7 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:17.358229 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-a45d42cf-1c50-48ed-87f7-d3414dc32cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:17.364434 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c7459f-1d8a-43f7-a3bb-fe5f729d26a8 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:17.366259 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 773/1554] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:17.367123 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d42cf-1c50-48ed-87f7-d3414dc32cf7 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:17.371631 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a9236a-08c6-487e-9b4b-3e623c8d765a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:17.372740 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a9236a-08c6-487e-9b4b-3e623c8d765a admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:17.373229 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a9236a-08c6-487e-9b4b-3e623c8d765a admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:17.373697 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a9236a-08c6-487e-9b4b-3e623c8d765a admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:17.374435 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a9236a-08c6-487e-9b4b-3e623c8d765a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:17.376157 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a9236a-08c6-487e-9b4b-3e623c8d765a admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:17.377575 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d42cf-1c50-48ed-87f7-d3414dc32cf7 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:17.382952 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b4ed58-0978-4bbf-bdd6-56f66878c9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:17.383616 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4ed58-0978-4bbf-bdd6-56f66878c9ad admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:17.383898 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4ed58-0978-4bbf-bdd6-56f66878c9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:17.384202 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4ed58-0978-4bbf-bdd6-56f66878c9ad admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:17.384923 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b4ed58-0978-4bbf-bdd6-56f66878c9ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:17.386707 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: WARNING py.warnings [None req-79b4ed58-0978-4bbf-bdd6-56f66878c9ad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 06 14:58:17.386707 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: warnings.warn(msg) May 06 14:58:17.386707 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: May 06 14:58:17.388116 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b4ed58-0978-4bbf-bdd6-56f66878c9ad admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:17.388769 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 782/1555] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:58:17.390341 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd9d9e-5cc6-424d-a9a6-ef9374c966b6 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:17.390474 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd9d9e-5cc6-424d-a9a6-ef9374c966b6 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:17.390701 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd9d9e-5cc6-424d-a9a6-ef9374c966b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:17.410561 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 774/1556] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:58:17 2022] POST /identity/v3/users => generated 288 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:58:17.413217 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-f5dd9d9e-5cc6-424d-a9a6-ef9374c966b6 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:17.414626 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f5dd9d9e-5cc6-424d-a9a6-ef9374c966b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:17.415457 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0abdd05-cb98-4f87-9867-318107c209a8 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:17.415756 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0abdd05-cb98-4f87-9867-318107c209a8 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:17.416116 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0abdd05-cb98-4f87-9867-318107c209a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:17.424398 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 783/1557] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:17.425831 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1757568-238c-4e0b-bcba-38eb07fab166 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:17.426121 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1757568-238c-4e0b-bcba-38eb07fab166 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:17.426448 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1757568-238c-4e0b-bcba-38eb07fab166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:17.434907 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-b0abdd05-cb98-4f87-9867-318107c209a8 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:17.436267 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b0abdd05-cb98-4f87-9867-318107c209a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:17.446472 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-a1757568-238c-4e0b-bcba-38eb07fab166 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:17.447650 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-a1757568-238c-4e0b-bcba-38eb07fab166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:17.450262 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 775/1558] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:17.451561 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ca2b6-7f73-4dc2-90d3-42014f3a4548 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:17.452190 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ca2b6-7f73-4dc2-90d3-42014f3a4548 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:17.452473 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ca2b6-7f73-4dc2-90d3-42014f3a4548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:17.467069 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 784/1559] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:17.467069 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ff00e-b666-4c77-88ae-0276eba3fb13 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:17.467069 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ff00e-b666-4c77-88ae-0276eba3fb13 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:17.467069 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ff00e-b666-4c77-88ae-0276eba3fb13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:17.474704 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-a74ca2b6-7f73-4dc2-90d3-42014f3a4548 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:17.475961 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-a74ca2b6-7f73-4dc2-90d3-42014f3a4548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:17.488466 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-af2ff00e-b666-4c77-88ae-0276eba3fb13 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:17.488683 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 776/1560] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:17.488772 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-af2ff00e-b666-4c77-88ae-0276eba3fb13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:17.488877 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909018f0-e1f5-4cf5-9346-0d7c79d91c4f None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:17.488877 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909018f0-e1f5-4cf5-9346-0d7c79d91c4f None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:17.488877 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909018f0-e1f5-4cf5-9346-0d7c79d91c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:17.503888 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 785/1561] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:17.505410 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ff00e-b666-4c77-88ae-0276eba3fb13 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:17.514183 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-909018f0-e1f5-4cf5-9346-0d7c79d91c4f None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:17.515378 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-909018f0-e1f5-4cf5-9346-0d7c79d91c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:17.519717 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ff00e-b666-4c77-88ae-0276eba3fb13 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:17.524526 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d9af48-dd3e-49e7-a263-336a9f920a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:17.525238 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9af48-dd3e-49e7-a263-336a9f920a44 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:17.525523 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9af48-dd3e-49e7-a263-336a9f920a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:17.525801 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9af48-dd3e-49e7-a263-336a9f920a44 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:17.529890 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 777/1562] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:17.530065 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d9af48-dd3e-49e7-a263-336a9f920a44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:17.530174 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909018f0-e1f5-4cf5-9346-0d7c79d91c4f None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:17.530264 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d9af48-dd3e-49e7-a263-336a9f920a44 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:17.542013 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 786/1563] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:58:17.542013 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d9af48-dd3e-49e7-a263-336a9f920a44 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:17.542381 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909018f0-e1f5-4cf5-9346-0d7c79d91c4f None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:17.549080 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb83c944-4f19-449e-b20c-52e0ff451327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:17.550067 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb83c944-4f19-449e-b20c-52e0ff451327 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:17.550501 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb83c944-4f19-449e-b20c-52e0ff451327 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:17.550932 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb83c944-4f19-449e-b20c-52e0ff451327 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:17.552101 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb83c944-4f19-449e-b20c-52e0ff451327 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:17.552416 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d9af48-dd3e-49e7-a263-336a9f920a44 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:17.554679 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb83c944-4f19-449e-b20c-52e0ff451327 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:17.557238 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e30a88-e0c0-4e0f-bbf3-6e7b199ba0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:17.558584 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e30a88-e0c0-4e0f-bbf3-6e7b199ba0d8 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:17.559482 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e30a88-e0c0-4e0f-bbf3-6e7b199ba0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:17.559769 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e30a88-e0c0-4e0f-bbf3-6e7b199ba0d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:17.560479 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e30a88-e0c0-4e0f-bbf3-6e7b199ba0d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:58:17.560821 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e30a88-e0c0-4e0f-bbf3-6e7b199ba0d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:17.562460 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e30a88-e0c0-4e0f-bbf3-6e7b199ba0d8 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:17.568566 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 778/1564] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:58:17.569941 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb83c944-4f19-449e-b20c-52e0ff451327 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:17.576170 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 787/1565] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 14:58:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:58:17.577493 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e30a88-e0c0-4e0f-bbf3-6e7b199ba0d8 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:17.581184 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb83c944-4f19-449e-b20c-52e0ff451327 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:17.586134 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cedf0d9-2b1a-4e02-ae4d-fa34c51bb8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:17.586788 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cedf0d9-2b1a-4e02-ae4d-fa34c51bb8f3 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:17.587069 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cedf0d9-2b1a-4e02-ae4d-fa34c51bb8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:17.587340 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cedf0d9-2b1a-4e02-ae4d-fa34c51bb8f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:17.587459 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e30a88-e0c0-4e0f-bbf3-6e7b199ba0d8 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:17.588060 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cedf0d9-2b1a-4e02-ae4d-fa34c51bb8f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:58:17.590466 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cedf0d9-2b1a-4e02-ae4d-fa34c51bb8f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:17.590466 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cedf0d9-2b1a-4e02-ae4d-fa34c51bb8f3 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:17.594371 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a9de0-e8a0-4ce2-9d29-a0312bdfe3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:17.595030 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a9de0-e8a0-4ce2-9d29-a0312bdfe3e8 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:17.595312 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a9de0-e8a0-4ce2-9d29-a0312bdfe3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:17.595585 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a9de0-e8a0-4ce2-9d29-a0312bdfe3e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:17.596303 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468a9de0-e8a0-4ce2-9d29-a0312bdfe3e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:58:17.596627 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468a9de0-e8a0-4ce2-9d29-a0312bdfe3e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:17.598225 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468a9de0-e8a0-4ce2-9d29-a0312bdfe3e8 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:17.608156 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 779/1566] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 14:58:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:58:17.609047 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cedf0d9-2b1a-4e02-ae4d-fa34c51bb8f3 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:17.610632 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 788/1567] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 14:58:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:58:17.612799 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a9de0-e8a0-4ce2-9d29-a0312bdfe3e8 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:17.624802 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a9de0-e8a0-4ce2-9d29-a0312bdfe3e8 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:17.627055 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cedf0d9-2b1a-4e02-ae4d-fa34c51bb8f3 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:17.629398 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7862b2a9-7461-4794-9050-457e0560e693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:17.630053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862b2a9-7461-4794-9050-457e0560e693 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:17.630397 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862b2a9-7461-4794-9050-457e0560e693 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:17.630724 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862b2a9-7461-4794-9050-457e0560e693 admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:17.631309 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7862b2a9-7461-4794-9050-457e0560e693 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:58:17.631636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7862b2a9-7461-4794-9050-457e0560e693 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:17.631890 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b65952b-9830-452a-8d25-e96defbea606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:17.632541 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65952b-9830-452a-8d25-e96defbea606 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:17.632855 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65952b-9830-452a-8d25-e96defbea606 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:17.633132 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65952b-9830-452a-8d25-e96defbea606 admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:17.633240 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7862b2a9-7461-4794-9050-457e0560e693 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:17.633843 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b65952b-9830-452a-8d25-e96defbea606 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:58:17.634151 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b65952b-9830-452a-8d25-e96defbea606 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:17.636442 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b65952b-9830-452a-8d25-e96defbea606 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:17.653791 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 789/1568] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 14:58:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:58:17.653791 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7862b2a9-7461-4794-9050-457e0560e693 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:17.654430 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 780/1569] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 14:58:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:58:17.654430 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b65952b-9830-452a-8d25-e96defbea606 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:17.664139 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7862b2a9-7461-4794-9050-457e0560e693 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:17.665376 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b65952b-9830-452a-8d25-e96defbea606 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:17.668819 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b11bd5-83ce-4296-a270-96ba6c1b61bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:17.669482 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b11bd5-83ce-4296-a270-96ba6c1b61bc admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:17.669765 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b11bd5-83ce-4296-a270-96ba6c1b61bc admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:17.670046 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b11bd5-83ce-4296-a270-96ba6c1b61bc admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:17.670768 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b11bd5-83ce-4296-a270-96ba6c1b61bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:58:17.671078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b11bd5-83ce-4296-a270-96ba6c1b61bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:17.672722 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b11bd5-83ce-4296-a270-96ba6c1b61bc admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:17.674129 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c410e617-7efa-4eb8-8ea2-8907326da4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:17.675708 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c410e617-7efa-4eb8-8ea2-8907326da4ae admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:17.675708 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c410e617-7efa-4eb8-8ea2-8907326da4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:17.675708 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c410e617-7efa-4eb8-8ea2-8907326da4ae admin admin] PATH_INFO: `/v3/projects/3cab862a707246328df1cfcad6ba10cc/users/a50983a75196418eb3f00328a1a60213/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:17.686804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 790/1570] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 14:58:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:58:17.688250 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b11bd5-83ce-4296-a270-96ba6c1b61bc admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:17.700035 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c410e617-7efa-4eb8-8ea2-8907326da4ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cab862a707246328df1cfcad6ba10cc, user_id=a50983a75196418eb3f00328a1a60213, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:17.702881 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b11bd5-83ce-4296-a270-96ba6c1b61bc admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:17.703012 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c410e617-7efa-4eb8-8ea2-8907326da4ae admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:17.706190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7dbfc4-e24e-42eb-9097-b9089bbe3cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:17.706865 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7dbfc4-e24e-42eb-9097-b9089bbe3cc3 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:17.707154 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7dbfc4-e24e-42eb-9097-b9089bbe3cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:17.707434 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7dbfc4-e24e-42eb-9097-b9089bbe3cc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:17.708374 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7dbfc4-e24e-42eb-9097-b9089bbe3cc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:17.710214 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7dbfc4-e24e-42eb-9097-b9089bbe3cc3 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:17.717370 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 781/1571] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:58:17 2022] PUT /identity/v3/projects/3cab862a707246328df1cfcad6ba10cc/users/a50983a75196418eb3f00328a1a60213/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:17.718108 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c410e617-7efa-4eb8-8ea2-8907326da4ae admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:17.722117 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 791/1572] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:58:17.722936 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7dbfc4-e24e-42eb-9097-b9089bbe3cc3 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:17.732626 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7dbfc4-e24e-42eb-9097-b9089bbe3cc3 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:17.782321 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c410e617-7efa-4eb8-8ea2-8907326da4ae admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:17.787300 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef881ca7-acbc-4ac2-890a-2f929b19b2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:17.787967 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef881ca7-acbc-4ac2-890a-2f929b19b2a6 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:17.788255 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef881ca7-acbc-4ac2-890a-2f929b19b2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:17.788530 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef881ca7-acbc-4ac2-890a-2f929b19b2a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:17.789247 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef881ca7-acbc-4ac2-890a-2f929b19b2a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:17.790036 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae9771-e7da-4ecc-9a52-dbf4a8a4cd5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:17.790698 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae9771-e7da-4ecc-9a52-dbf4a8a4cd5e admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:17.791033 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae9771-e7da-4ecc-9a52-dbf4a8a4cd5e admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:17.791143 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef881ca7-acbc-4ac2-890a-2f929b19b2a6 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:17.791319 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae9771-e7da-4ecc-9a52-dbf4a8a4cd5e admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:17.792238 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ae9771-e7da-4ecc-9a52-dbf4a8a4cd5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:17.794337 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ae9771-e7da-4ecc-9a52-dbf4a8a4cd5e admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:17.821415 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 782/1573] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:58:17 2022] POST /identity/v3/projects => generated 389 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:58:17.821415 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef881ca7-acbc-4ac2-890a-2f929b19b2a6 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:17.827175 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 792/1574] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:58:17 2022] POST /identity/v3/projects => generated 393 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:58:17.827175 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae9771-e7da-4ecc-9a52-dbf4a8a4cd5e admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:17.883229 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef881ca7-acbc-4ac2-890a-2f929b19b2a6 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:17.887904 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e844026b-c2da-41b7-b5f4-cc2d938b5b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:17.888570 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844026b-c2da-41b7-b5f4-cc2d938b5b1c admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:17.888889 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844026b-c2da-41b7-b5f4-cc2d938b5b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:17.889167 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844026b-c2da-41b7-b5f4-cc2d938b5b1c admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:17.889394 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae9771-e7da-4ecc-9a52-dbf4a8a4cd5e admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:17.889855 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e844026b-c2da-41b7-b5f4-cc2d938b5b1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:17.891819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e844026b-c2da-41b7-b5f4-cc2d938b5b1c admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:17.893989 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb48797a-2475-49bc-bc96-663c9a92ad96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:17.894641 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb48797a-2475-49bc-bc96-663c9a92ad96 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:17.894923 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb48797a-2475-49bc-bc96-663c9a92ad96 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:17.895197 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb48797a-2475-49bc-bc96-663c9a92ad96 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:17.895917 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb48797a-2475-49bc-bc96-663c9a92ad96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:17.897839 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb48797a-2475-49bc-bc96-663c9a92ad96 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:17.919239 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 783/1575] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:58:17 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:58:17.920014 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e844026b-c2da-41b7-b5f4-cc2d938b5b1c admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:17.930757 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 793/1576] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:58:17 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:58:17.931570 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb48797a-2475-49bc-bc96-663c9a92ad96 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:17.984446 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e844026b-c2da-41b7-b5f4-cc2d938b5b1c admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:18.002576 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb48797a-2475-49bc-bc96-663c9a92ad96 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:18.008239 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b493c3fe-1258-44bd-af38-4b1a9dfe63e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:18.009112 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493c3fe-1258-44bd-af38-4b1a9dfe63e6 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:18.009401 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493c3fe-1258-44bd-af38-4b1a9dfe63e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:18.009690 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493c3fe-1258-44bd-af38-4b1a9dfe63e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:18.010583 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b493c3fe-1258-44bd-af38-4b1a9dfe63e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:18.012734 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b493c3fe-1258-44bd-af38-4b1a9dfe63e6 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:18.043692 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 794/1577] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:58:17 2022] POST /identity/v3/projects => generated 367 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:58:18.045163 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c541c4-8039-4a09-a36c-192cbdbad449 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:18.045410 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758fe89f-67cd-489c-be5c-9177e85fc12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:18.045567 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c541c4-8039-4a09-a36c-192cbdbad449 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:18.045770 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c541c4-8039-4a09-a36c-192cbdbad449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:18.046127 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758fe89f-67cd-489c-be5c-9177e85fc12f admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:18.046409 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758fe89f-67cd-489c-be5c-9177e85fc12f admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:18.046685 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758fe89f-67cd-489c-be5c-9177e85fc12f admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:18.047645 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758fe89f-67cd-489c-be5c-9177e85fc12f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:18.049652 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758fe89f-67cd-489c-be5c-9177e85fc12f admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:18.063856 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-27c541c4-8039-4a09-a36c-192cbdbad449 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:18.065088 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-27c541c4-8039-4a09-a36c-192cbdbad449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:18.077174 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 784/1578] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:58:17 2022] POST /identity/v3/projects => generated 405 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:58:18.077174 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758fe89f-67cd-489c-be5c-9177e85fc12f admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:18.135410 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 795/1579] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:18.137303 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c541c4-8039-4a09-a36c-192cbdbad449 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:18.145010 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758fe89f-67cd-489c-be5c-9177e85fc12f admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:18.146929 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c541c4-8039-4a09-a36c-192cbdbad449 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:18.149751 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f3b44b-9c9d-4dd5-bfcc-090ba2886c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:18.150644 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f3b44b-9c9d-4dd5-bfcc-090ba2886c4c admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:18.150924 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f3b44b-9c9d-4dd5-bfcc-090ba2886c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:18.151250 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f3b44b-9c9d-4dd5-bfcc-090ba2886c4c admin admin] PATH_INFO: `/v3/projects/5ad13dae3f6649fb8d8d2bdd1f1caaef/users/afd9f5519134415b8fc8a8c08149c502/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:18.152364 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d919f90-ae94-4d11-9807-ef8b9491a235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:18.153177 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d919f90-ae94-4d11-9807-ef8b9491a235 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:18.153563 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d919f90-ae94-4d11-9807-ef8b9491a235 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:18.153746 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d919f90-ae94-4d11-9807-ef8b9491a235 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:18.154502 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d919f90-ae94-4d11-9807-ef8b9491a235 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:18.156882 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d919f90-ae94-4d11-9807-ef8b9491a235 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:18.171975 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f3b44b-9c9d-4dd5-bfcc-090ba2886c4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ad13dae3f6649fb8d8d2bdd1f1caaef, user_id=afd9f5519134415b8fc8a8c08149c502, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:18.174625 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f3b44b-9c9d-4dd5-bfcc-090ba2886c4c admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:18.177528 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 796/1580] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:58:18 2022] POST /identity/v3/users => generated 295 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:58:18.178546 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d919f90-ae94-4d11-9807-ef8b9491a235 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:18.191759 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d919f90-ae94-4d11-9807-ef8b9491a235 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:18.192006 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 785/1581] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:58:18 2022] PUT /identity/v3/projects/5ad13dae3f6649fb8d8d2bdd1f1caaef/users/afd9f5519134415b8fc8a8c08149c502/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:18.195701 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f3b44b-9c9d-4dd5-bfcc-090ba2886c4c admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:18.256078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903e77f-0bba-474b-a784-2c880d6d344e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:18.256788 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903e77f-0bba-474b-a784-2c880d6d344e admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:18.257073 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903e77f-0bba-474b-a784-2c880d6d344e admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:18.257347 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903e77f-0bba-474b-a784-2c880d6d344e admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:18.258029 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903e77f-0bba-474b-a784-2c880d6d344e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:18.259173 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f3b44b-9c9d-4dd5-bfcc-090ba2886c4c admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:18.259926 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903e77f-0bba-474b-a784-2c880d6d344e admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:18.268150 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c46b6-f2d6-40b0-b8c4-ab759a68240e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:18.268873 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c46b6-f2d6-40b0-b8c4-ab759a68240e admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:18.269198 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c46b6-f2d6-40b0-b8c4-ab759a68240e admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:18.269402 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c46b6-f2d6-40b0-b8c4-ab759a68240e admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:18.270076 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c46b6-f2d6-40b0-b8c4-ab759a68240e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:18.272200 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c46b6-f2d6-40b0-b8c4-ab759a68240e admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:18.283882 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 797/1582] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:58:18 2022] POST /identity/v3/users => generated 297 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:58:18.285293 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903e77f-0bba-474b-a784-2c880d6d344e admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:18.293901 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 786/1583] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:58:18 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:58:18.294909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c46b6-f2d6-40b0-b8c4-ab759a68240e admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:18.297838 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903e77f-0bba-474b-a784-2c880d6d344e admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:18.304007 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd4c62b-7016-45d0-8820-5034ce0fd1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:18.304721 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4c62b-7016-45d0-8820-5034ce0fd1d1 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:18.305008 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4c62b-7016-45d0-8820-5034ce0fd1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:18.305277 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4c62b-7016-45d0-8820-5034ce0fd1d1 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:18.305965 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd4c62b-7016-45d0-8820-5034ce0fd1d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:18.306403 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c46b6-f2d6-40b0-b8c4-ab759a68240e admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:18.308710 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd4c62b-7016-45d0-8820-5034ce0fd1d1 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:18.311918 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0afb9da-8598-4cba-a3d9-7457854dd637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:18.312585 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0afb9da-8598-4cba-a3d9-7457854dd637 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:18.312903 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0afb9da-8598-4cba-a3d9-7457854dd637 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:18.313179 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0afb9da-8598-4cba-a3d9-7457854dd637 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:18.313873 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0afb9da-8598-4cba-a3d9-7457854dd637 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:18.315850 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0afb9da-8598-4cba-a3d9-7457854dd637 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:18.331277 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 798/1584] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:58:18 2022] POST /identity/v3/users => generated 287 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:58:18.332399 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd4c62b-7016-45d0-8820-5034ce0fd1d1 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:18.343425 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 787/1585] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:58:18 2022] POST /identity/v3/users => generated 284 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:58:18.343425 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0afb9da-8598-4cba-a3d9-7457854dd637 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:18.343709 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd4c62b-7016-45d0-8820-5034ce0fd1d1 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:18.347023 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772db9d2-7573-4705-ab9c-aade6525cc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:18.347681 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772db9d2-7573-4705-ab9c-aade6525cc02 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:18.347963 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772db9d2-7573-4705-ab9c-aade6525cc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:18.348237 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772db9d2-7573-4705-ab9c-aade6525cc02 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:18.349261 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772db9d2-7573-4705-ab9c-aade6525cc02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:18.351202 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772db9d2-7573-4705-ab9c-aade6525cc02 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:18.372097 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 799/1586] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:58:18 2022] POST /identity/v3/users => generated 303 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:58:18.372955 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772db9d2-7573-4705-ab9c-aade6525cc02 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:18.382498 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772db9d2-7573-4705-ab9c-aade6525cc02 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:18.388018 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb0eea8-565d-4bad-ba27-a8bea7950e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:18.388716 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb0eea8-565d-4bad-ba27-a8bea7950e60 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:18.389001 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb0eea8-565d-4bad-ba27-a8bea7950e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:18.389277 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb0eea8-565d-4bad-ba27-a8bea7950e60 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:18.390007 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb0eea8-565d-4bad-ba27-a8bea7950e60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:18.391594 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb0eea8-565d-4bad-ba27-a8bea7950e60 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:18.400043 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0afb9da-8598-4cba-a3d9-7457854dd637 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:18.401750 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 800/1587] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:58:18.403050 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5d9dd-9e08-46db-b306-43965dc3d3d7 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:18.403321 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5d9dd-9e08-46db-b306-43965dc3d3d7 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:18.403596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5d9dd-9e08-46db-b306-43965dc3d3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:18.407988 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cd1717-f39c-4c1b-b78f-4b57ba7336d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:18.408838 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd1717-f39c-4c1b-b78f-4b57ba7336d1 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:18.409167 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd1717-f39c-4c1b-b78f-4b57ba7336d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:18.409457 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd1717-f39c-4c1b-b78f-4b57ba7336d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:18.420685 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cd1717-f39c-4c1b-b78f-4b57ba7336d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:18.421948 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-34d5d9dd-9e08-46db-b306-43965dc3d3d7 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:18.422200 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cd1717-f39c-4c1b-b78f-4b57ba7336d1 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:18.423163 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-34d5d9dd-9e08-46db-b306-43965dc3d3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:18.431166 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 801/1588] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:18.431833 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5d9dd-9e08-46db-b306-43965dc3d3d7 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:18.433843 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 788/1589] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:58:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:58:18.435214 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cd1717-f39c-4c1b-b78f-4b57ba7336d1 service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:18.443457 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5d9dd-9e08-46db-b306-43965dc3d3d7 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:18.446174 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cd1717-f39c-4c1b-b78f-4b57ba7336d1 service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:18.449729 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1756bdfa-8720-4b7d-820e-8a6c04214048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:18.450214 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756bdfa-8720-4b7d-820e-8a6c04214048 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:18.450571 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756bdfa-8720-4b7d-820e-8a6c04214048 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:18.450952 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756bdfa-8720-4b7d-820e-8a6c04214048 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:18.451935 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1756bdfa-8720-4b7d-820e-8a6c04214048 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:18.452773 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3de4e5-0ac3-4fc4-bef0-7106bc935246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:18.453509 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3de4e5-0ac3-4fc4-bef0-7106bc935246 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:18.453700 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1756bdfa-8720-4b7d-820e-8a6c04214048 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:18.453929 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3de4e5-0ac3-4fc4-bef0-7106bc935246 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:18.454231 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3de4e5-0ac3-4fc4-bef0-7106bc935246 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:18.455069 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3de4e5-0ac3-4fc4-bef0-7106bc935246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:18.456765 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3de4e5-0ac3-4fc4-bef0-7106bc935246 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:18.467672 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 802/1590] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:58:18.469074 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1756bdfa-8720-4b7d-820e-8a6c04214048 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:18.469333 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 789/1591] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:58:18.470181 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3de4e5-0ac3-4fc4-bef0-7106bc935246 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:18.481709 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3de4e5-0ac3-4fc4-bef0-7106bc935246 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:18.485439 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1756bdfa-8720-4b7d-820e-8a6c04214048 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:18.486824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19276c8-7b16-4d85-8bdc-943b8a32ca27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:18.487573 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19276c8-7b16-4d85-8bdc-943b8a32ca27 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:18.487855 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19276c8-7b16-4d85-8bdc-943b8a32ca27 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:18.488129 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19276c8-7b16-4d85-8bdc-943b8a32ca27 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:18.488875 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19276c8-7b16-4d85-8bdc-943b8a32ca27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:18.490552 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19276c8-7b16-4d85-8bdc-943b8a32ca27 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:18.491061 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a31302-f1ff-4005-af12-6fd4f44be082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:18.491881 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a31302-f1ff-4005-af12-6fd4f44be082 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:18.492072 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a31302-f1ff-4005-af12-6fd4f44be082 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:18.492811 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a31302-f1ff-4005-af12-6fd4f44be082 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:18.493106 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a31302-f1ff-4005-af12-6fd4f44be082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:18.494774 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a31302-f1ff-4005-af12-6fd4f44be082 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:18.504089 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 790/1592] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:58:18.505487 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19276c8-7b16-4d85-8bdc-943b8a32ca27 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:18.506536 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 803/1593] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:18 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:58:18.507827 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a31302-f1ff-4005-af12-6fd4f44be082 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:18.516800 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19276c8-7b16-4d85-8bdc-943b8a32ca27 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:18.518186 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a31302-f1ff-4005-af12-6fd4f44be082 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:18.522070 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f2ec6d-cfcb-4047-9702-64c4560ef23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:18.522871 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f2ec6d-cfcb-4047-9702-64c4560ef23d admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:18.523157 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f2ec6d-cfcb-4047-9702-64c4560ef23d admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:18.523428 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f2ec6d-cfcb-4047-9702-64c4560ef23d admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:18.524150 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f2ec6d-cfcb-4047-9702-64c4560ef23d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:18.525918 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f2ec6d-cfcb-4047-9702-64c4560ef23d admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:18.533162 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df81be1-a1b5-4c39-b080-e68977e735fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:18.533837 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df81be1-a1b5-4c39-b080-e68977e735fc admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:18.534123 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df81be1-a1b5-4c39-b080-e68977e735fc admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:18.534400 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df81be1-a1b5-4c39-b080-e68977e735fc admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:18.535139 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df81be1-a1b5-4c39-b080-e68977e735fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:18.536847 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df81be1-a1b5-4c39-b080-e68977e735fc admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:18.539560 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 791/1594] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:58:18.540200 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f2ec6d-cfcb-4047-9702-64c4560ef23d admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:18.546003 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-a8f2ec6d-cfcb-4047-9702-64c4560ef23d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:18.556697 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 804/1595] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:18 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:58:18.556697 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df81be1-a1b5-4c39-b080-e68977e735fc admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:18.565329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df81be1-a1b5-4c39-b080-e68977e735fc admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:18.566466 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f2ec6d-cfcb-4047-9702-64c4560ef23d admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:18.570271 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0a209e-7baa-4e6c-955e-362683a3c822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:18.570860 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3badfa4-b6ac-4645-9287-9ed98a8eced5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:18.571028 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a209e-7baa-4e6c-955e-362683a3c822 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:18.571230 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a209e-7baa-4e6c-955e-362683a3c822 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:18.571511 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a209e-7baa-4e6c-955e-362683a3c822 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:18.572237 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0a209e-7baa-4e6c-955e-362683a3c822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:18.572488 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3badfa4-b6ac-4645-9287-9ed98a8eced5 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:18.572839 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3badfa4-b6ac-4645-9287-9ed98a8eced5 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:18.573776 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3badfa4-b6ac-4645-9287-9ed98a8eced5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:18.574428 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0a209e-7baa-4e6c-955e-362683a3c822 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:18.584550 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3badfa4-b6ac-4645-9287-9ed98a8eced5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:18.585492 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 805/1596] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:58:18.586373 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0a209e-7baa-4e6c-955e-362683a3c822 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:18.586863 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3badfa4-b6ac-4645-9287-9ed98a8eced5 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:18.599762 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0a209e-7baa-4e6c-955e-362683a3c822 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:18.600829 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 792/1597] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:58:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:58:18.601685 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3badfa4-b6ac-4645-9287-9ed98a8eced5 service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:18.605997 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c0251-6cd8-4e8b-9291-f4793840983f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:18.607190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c0251-6cd8-4e8b-9291-f4793840983f admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:18.608847 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c0251-6cd8-4e8b-9291-f4793840983f admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:18.609322 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c0251-6cd8-4e8b-9291-f4793840983f admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:18.610252 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560c0251-6cd8-4e8b-9291-f4793840983f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:18.612172 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3badfa4-b6ac-4645-9287-9ed98a8eced5 service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:18.612404 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560c0251-6cd8-4e8b-9291-f4793840983f admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:18.618987 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92beffa-5c57-40f7-8bba-770ed0a8bbde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:18.619664 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92beffa-5c57-40f7-8bba-770ed0a8bbde admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:18.620127 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92beffa-5c57-40f7-8bba-770ed0a8bbde admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:18.620416 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92beffa-5c57-40f7-8bba-770ed0a8bbde admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:18.621419 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92beffa-5c57-40f7-8bba-770ed0a8bbde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:18.624319 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92beffa-5c57-40f7-8bba-770ed0a8bbde admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:18.626125 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 806/1598] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:18 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:58:18.627470 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c0251-6cd8-4e8b-9291-f4793840983f admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:18.639623 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 793/1599] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:18 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:58:18.641826 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92beffa-5c57-40f7-8bba-770ed0a8bbde admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:18.644416 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c0251-6cd8-4e8b-9291-f4793840983f admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:18.656594 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92beffa-5c57-40f7-8bba-770ed0a8bbde admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:18.658336 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5a5afc-dcb9-42d0-b8a2-a8621546481c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:18.659764 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a5afc-dcb9-42d0-b8a2-a8621546481c admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:18.660119 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a5afc-dcb9-42d0-b8a2-a8621546481c admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:18.660431 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a5afc-dcb9-42d0-b8a2-a8621546481c admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:18.661265 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5a5afc-dcb9-42d0-b8a2-a8621546481c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:18.663066 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5a5afc-dcb9-42d0-b8a2-a8621546481c admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:18.663382 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95815a5c-f3c6-4e1d-8a11-5ca16a98f0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:18.664223 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95815a5c-f3c6-4e1d-8a11-5ca16a98f0d2 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:18.664678 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95815a5c-f3c6-4e1d-8a11-5ca16a98f0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:18.665076 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95815a5c-f3c6-4e1d-8a11-5ca16a98f0d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:18.665961 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95815a5c-f3c6-4e1d-8a11-5ca16a98f0d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:18.668078 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95815a5c-f3c6-4e1d-8a11-5ca16a98f0d2 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:18.677209 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 807/1600] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:18 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:58:18.677209 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5a5afc-dcb9-42d0-b8a2-a8621546481c admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:18.686886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 794/1601] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:18 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:58:18.686886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95815a5c-f3c6-4e1d-8a11-5ca16a98f0d2 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:18.693845 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5a5afc-dcb9-42d0-b8a2-a8621546481c admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:18.696414 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95815a5c-f3c6-4e1d-8a11-5ca16a98f0d2 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:18.700802 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bafe7d8-935c-4073-b20d-690b43647f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:18.702201 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bafe7d8-935c-4073-b20d-690b43647f23 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:18.702761 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bafe7d8-935c-4073-b20d-690b43647f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:18.702998 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e348e2-3e85-4cb1-9d9a-736aef40ba65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:18.703399 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bafe7d8-935c-4073-b20d-690b43647f23 admin admin] PATH_INFO: `/v3/projects/04793e5505aa4d32ab3efed2892ea2b4/users/02b07e776b2c4bfba6eabb15999aec35/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:18.704151 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e348e2-3e85-4cb1-9d9a-736aef40ba65 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:18.704536 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e348e2-3e85-4cb1-9d9a-736aef40ba65 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:18.704949 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e348e2-3e85-4cb1-9d9a-736aef40ba65 admin admin] PATH_INFO: `/v3/projects/66121d02dbff4c3a920161e445959085/users/81c9e87c348c4a2c82e8327a2cadfc5d/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:18.721473 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bafe7d8-935c-4073-b20d-690b43647f23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04793e5505aa4d32ab3efed2892ea2b4, user_id=02b07e776b2c4bfba6eabb15999aec35, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:18.722979 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: WARNING py.warnings [None req-9bafe7d8-935c-4073-b20d-690b43647f23 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 06 14:58:18.722979 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: warnings.warn(msg) May 06 14:58:18.722979 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: May 06 14:58:18.723524 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e348e2-3e85-4cb1-9d9a-736aef40ba65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66121d02dbff4c3a920161e445959085, user_id=81c9e87c348c4a2c82e8327a2cadfc5d, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:18.725913 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bafe7d8-935c-4073-b20d-690b43647f23 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:18.726784 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e348e2-3e85-4cb1-9d9a-736aef40ba65 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:18.744434 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 808/1602] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:58:18 2022] PUT /identity/v3/projects/04793e5505aa4d32ab3efed2892ea2b4/users/02b07e776b2c4bfba6eabb15999aec35/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:18.749250 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bafe7d8-935c-4073-b20d-690b43647f23 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:18.751549 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 795/1603] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:58:18 2022] PUT /identity/v3/projects/66121d02dbff4c3a920161e445959085/users/81c9e87c348c4a2c82e8327a2cadfc5d/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:18.754716 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e348e2-3e85-4cb1-9d9a-736aef40ba65 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:18.846582 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bafe7d8-935c-4073-b20d-690b43647f23 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:18.855132 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fbe6d2-6330-41bc-b9d8-71064292ef85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:18.855343 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e348e2-3e85-4cb1-9d9a-736aef40ba65 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:18.859716 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fbe6d2-6330-41bc-b9d8-71064292ef85 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:18.859716 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fbe6d2-6330-41bc-b9d8-71064292ef85 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:18.859716 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fbe6d2-6330-41bc-b9d8-71064292ef85 admin admin] PATH_INFO: `/v3/projects/09c7b905c6444fa3949c574f6084fead/users/bd41a5b586e04553bc0fa64c2a8b1ba1/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:18.867001 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc7703-097e-4a00-a3bf-5d12d8cc8a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:18.868101 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc7703-097e-4a00-a3bf-5d12d8cc8a73 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:18.868486 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc7703-097e-4a00-a3bf-5d12d8cc8a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:18.868856 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc7703-097e-4a00-a3bf-5d12d8cc8a73 admin admin] PATH_INFO: `/v3/projects/30792c0ea24845fdb65dcd2b282d64ec/users/27fff5cf15794969b325cedffa2a90e5/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:18.881502 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fbe6d2-6330-41bc-b9d8-71064292ef85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09c7b905c6444fa3949c574f6084fead, user_id=bd41a5b586e04553bc0fa64c2a8b1ba1, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:18.887511 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fbe6d2-6330-41bc-b9d8-71064292ef85 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:18.899076 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cc7703-097e-4a00-a3bf-5d12d8cc8a73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30792c0ea24845fdb65dcd2b282d64ec, user_id=27fff5cf15794969b325cedffa2a90e5, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:18.902500 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cc7703-097e-4a00-a3bf-5d12d8cc8a73 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:18.920524 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 809/1604] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:58:18 2022] PUT /identity/v3/projects/09c7b905c6444fa3949c574f6084fead/users/bd41a5b586e04553bc0fa64c2a8b1ba1/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:18.920524 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fbe6d2-6330-41bc-b9d8-71064292ef85 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:18.923401 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 796/1605] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:58:18 2022] PUT /identity/v3/projects/30792c0ea24845fdb65dcd2b282d64ec/users/27fff5cf15794969b325cedffa2a90e5/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:18.935648 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc7703-097e-4a00-a3bf-5d12d8cc8a73 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:19.032594 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc7703-097e-4a00-a3bf-5d12d8cc8a73 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:19.036930 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fbe6d2-6330-41bc-b9d8-71064292ef85 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:19.038202 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493b5735-1a36-4e52-b260-4880405607c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:19.039253 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b5735-1a36-4e52-b260-4880405607c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:19.039541 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b5735-1a36-4e52-b260-4880405607c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:19.040109 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b5735-1a36-4e52-b260-4880405607c6 admin admin] PATH_INFO: `/v3/projects/4637699c99024ca2a25d4afed19102d2/users/684316ea5603441a93c5390572f7fbd0/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:19.046249 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2b58e8-649e-42f0-8bb0-b190f61a5961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:19.047385 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2b58e8-649e-42f0-8bb0-b190f61a5961 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:19.047771 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2b58e8-649e-42f0-8bb0-b190f61a5961 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:19.048147 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2b58e8-649e-42f0-8bb0-b190f61a5961 admin admin] PATH_INFO: `/v3/projects/30209268284f4cf791a88f4b623e4d1f/users/4c56619e7cb145f5930e46270571c828/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:19.060454 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493b5735-1a36-4e52-b260-4880405607c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4637699c99024ca2a25d4afed19102d2, user_id=684316ea5603441a93c5390572f7fbd0, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:19.063702 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493b5735-1a36-4e52-b260-4880405607c6 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:19.071983 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2b58e8-649e-42f0-8bb0-b190f61a5961 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30209268284f4cf791a88f4b623e4d1f, user_id=4c56619e7cb145f5930e46270571c828, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:19.077693 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2b58e8-649e-42f0-8bb0-b190f61a5961 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:19.085750 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 797/1606] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:58:18 2022] PUT /identity/v3/projects/4637699c99024ca2a25d4afed19102d2/users/684316ea5603441a93c5390572f7fbd0/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:19.092808 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dfeb86-0ced-49aa-be0d-6df208950cd8 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:19.093111 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dfeb86-0ced-49aa-be0d-6df208950cd8 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:19.093370 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dfeb86-0ced-49aa-be0d-6df208950cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:19.106916 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 810/1607] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:58:18 2022] PUT /identity/v3/projects/30209268284f4cf791a88f4b623e4d1f/users/4c56619e7cb145f5930e46270571c828/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:19.108484 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40cf0b-b21b-4302-a3e0-0c46e0d070e7 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:19.108815 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40cf0b-b21b-4302-a3e0-0c46e0d070e7 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:19.109097 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40cf0b-b21b-4302-a3e0-0c46e0d070e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:19.118955 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-98dfeb86-0ced-49aa-be0d-6df208950cd8 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:19.121163 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-98dfeb86-0ced-49aa-be0d-6df208950cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:19.137330 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-4c40cf0b-b21b-4302-a3e0-0c46e0d070e7 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:19.138646 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-4c40cf0b-b21b-4302-a3e0-0c46e0d070e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:19.203716 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 798/1608] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:19.206691 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aad7fe4-0d9c-436f-aeb2-1fcc65dc93b6 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:19.207095 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aad7fe4-0d9c-436f-aeb2-1fcc65dc93b6 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:19.207272 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aad7fe4-0d9c-436f-aeb2-1fcc65dc93b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:19.222752 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 811/1609] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:19.226883 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fce743-3366-4cee-a9b0-9b4e63a15ea8 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:19.227175 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fce743-3366-4cee-a9b0-9b4e63a15ea8 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:19.227454 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fce743-3366-4cee-a9b0-9b4e63a15ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:19.242213 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-6aad7fe4-0d9c-436f-aeb2-1fcc65dc93b6 None None] MFA Rules not processed for user `bd41a5b586e04553bc0fa64c2a8b1ba1`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:19.243655 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-6aad7fe4-0d9c-436f-aeb2-1fcc65dc93b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:19.255785 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-d9fce743-3366-4cee-a9b0-9b4e63a15ea8 None None] MFA Rules not processed for user `27fff5cf15794969b325cedffa2a90e5`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:19.257217 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-d9fce743-3366-4cee-a9b0-9b4e63a15ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:19.316734 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 799/1610] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:19.318547 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2031c40-71bd-48b2-bd10-30f4f87d85ae None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:19.318912 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2031c40-71bd-48b2-bd10-30f4f87d85ae None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:19.319641 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2031c40-71bd-48b2-bd10-30f4f87d85ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:19.337821 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-c2031c40-71bd-48b2-bd10-30f4f87d85ae None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:19.339168 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-c2031c40-71bd-48b2-bd10-30f4f87d85ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:19.340725 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 812/1611] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:19 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:19.342202 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b812a74-94f0-4e35-8daf-ef759579fd47 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:19.342567 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b812a74-94f0-4e35-8daf-ef759579fd47 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:19.342917 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b812a74-94f0-4e35-8daf-ef759579fd47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:19.356453 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 800/1612] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:19.356453 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2031c40-71bd-48b2-bd10-30f4f87d85ae None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:19.373432 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-6b812a74-94f0-4e35-8daf-ef759579fd47 None None] MFA Rules not processed for user `4c56619e7cb145f5930e46270571c828`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:19.374641 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-6b812a74-94f0-4e35-8daf-ef759579fd47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:19.422910 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2031c40-71bd-48b2-bd10-30f4f87d85ae None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:19.427829 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b772e-a0da-4d50-8d1b-7dd4b51ec760 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:19.428489 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b772e-a0da-4d50-8d1b-7dd4b51ec760 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:19.428869 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b772e-a0da-4d50-8d1b-7dd4b51ec760 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:19.429148 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b772e-a0da-4d50-8d1b-7dd4b51ec760 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:19.440663 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79b772e-a0da-4d50-8d1b-7dd4b51ec760 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:19.442119 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79b772e-a0da-4d50-8d1b-7dd4b51ec760 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:19.452222 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 813/1613] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:19.453114 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b812a74-94f0-4e35-8daf-ef759579fd47 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:19.453647 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 801/1614] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:58:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:58:19.455040 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d0e01-494f-4d73-991f-77d125eb78c0 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:19.455341 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d0e01-494f-4d73-991f-77d125eb78c0 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:19.455689 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d0e01-494f-4d73-991f-77d125eb78c0 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:19.457356 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 802/1615] 10.222.0.23 () {54 vars in 960 bytes} [Fri May 6 14:58:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:58:19.458172 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d0e01-494f-4d73-991f-77d125eb78c0 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:19.488294 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b812a74-94f0-4e35-8daf-ef759579fd47 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:19.493954 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec73ce-d1a5-4e30-833f-cf6086ebb88c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:19.494620 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec73ce-d1a5-4e30-833f-cf6086ebb88c service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:19.495529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec73ce-d1a5-4e30-833f-cf6086ebb88c service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:19.496480 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec73ce-d1a5-4e30-833f-cf6086ebb88c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:19.507014 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ec73ce-d1a5-4e30-833f-cf6086ebb88c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:19.509053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ec73ce-d1a5-4e30-833f-cf6086ebb88c service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:19.521653 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 814/1616] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:58:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:58:19.522330 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec73ce-d1a5-4e30-833f-cf6086ebb88c service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:19.526791 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d0e01-494f-4d73-991f-77d125eb78c0 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:19.532095 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b7fb5-e4b5-493d-a29a-9ad19c2f2f45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:19.533097 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b7fb5-e4b5-493d-a29a-9ad19c2f2f45 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:19.533241 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec73ce-d1a5-4e30-833f-cf6086ebb88c service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:19.534244 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b7fb5-e4b5-493d-a29a-9ad19c2f2f45 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:19.534714 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b7fb5-e4b5-493d-a29a-9ad19c2f2f45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:19.538090 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436257d3-4ccd-4d18-ad75-b872e1f91a80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:19.539652 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436257d3-4ccd-4d18-ad75-b872e1f91a80 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:19.540195 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436257d3-4ccd-4d18-ad75-b872e1f91a80 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:19.540648 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436257d3-4ccd-4d18-ad75-b872e1f91a80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:19.549561 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094b7fb5-e4b5-493d-a29a-9ad19c2f2f45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:19.553544 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436257d3-4ccd-4d18-ad75-b872e1f91a80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:19.554339 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094b7fb5-e4b5-493d-a29a-9ad19c2f2f45 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:19.555536 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436257d3-4ccd-4d18-ad75-b872e1f91a80 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:19.575035 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 803/1617] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:58:19 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:58:19.576166 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b7fb5-e4b5-493d-a29a-9ad19c2f2f45 service cinder] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:19.583530 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 815/1618] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:58:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:58:19.585773 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df572c-f762-4817-b910-011b63285453 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:19.586140 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df572c-f762-4817-b910-011b63285453 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:19.586983 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df572c-f762-4817-b910-011b63285453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:19.592355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b7fb5-e4b5-493d-a29a-9ad19c2f2f45 service cinder] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:19.611523 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09d0171-3ae0-427b-83f3-e3bed3490bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:19.612281 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d0171-3ae0-427b-83f3-e3bed3490bdf admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:19.612691 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d0171-3ae0-427b-83f3-e3bed3490bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:19.613080 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d0171-3ae0-427b-83f3-e3bed3490bdf admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:19.615147 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09d0171-3ae0-427b-83f3-e3bed3490bdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:19.617509 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09d0171-3ae0-427b-83f3-e3bed3490bdf admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:19.620357 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-81df572c-f762-4817-b910-011b63285453 None None] MFA Rules not processed for user `fc8cf09642ec4e10a80badcf8b585ef8`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:19.622103 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-81df572c-f762-4817-b910-011b63285453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:19.666298 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 804/1619] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:58:19 2022] POST /identity/v3/projects => generated 371 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:58:19.668946 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d406029-5069-4709-9c8c-974c390a91d1 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:19.669485 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d406029-5069-4709-9c8c-974c390a91d1 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:19.670180 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d406029-5069-4709-9c8c-974c390a91d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:19.704418 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-7d406029-5069-4709-9c8c-974c390a91d1 None None] MFA Rules not processed for user `a50983a75196418eb3f00328a1a60213`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:19.705816 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-7d406029-5069-4709-9c8c-974c390a91d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:19.770597 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 816/1620] 10.222.0.23 () {58 vars in 1057 bytes} [Fri May 6 14:58:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:19.773456 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81df572c-f762-4817-b910-011b63285453 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:19.797172 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 805/1621] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:19.810358 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d406029-5069-4709-9c8c-974c390a91d1 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:19.814365 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81df572c-f762-4817-b910-011b63285453 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:19.946069 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd344701-58d9-47c7-8683-ad2f0cad825e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:19.946784 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd344701-58d9-47c7-8683-ad2f0cad825e admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:19.947573 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd344701-58d9-47c7-8683-ad2f0cad825e admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:19.947855 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd344701-58d9-47c7-8683-ad2f0cad825e admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:19.952356 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd344701-58d9-47c7-8683-ad2f0cad825e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:19.956726 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd344701-58d9-47c7-8683-ad2f0cad825e admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:19.973635 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d406029-5069-4709-9c8c-974c390a91d1 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:19.987385 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda10710-4bc4-4dcb-94ba-4de79099633e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:19.989161 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda10710-4bc4-4dcb-94ba-4de79099633e service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:19.992550 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda10710-4bc4-4dcb-94ba-4de79099633e service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:19.992929 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda10710-4bc4-4dcb-94ba-4de79099633e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:20.000449 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 817/1622] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:58:19 2022] POST /identity/v3/users => generated 292 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:58:20.002492 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0bf1e-69f4-4fd0-affd-8f170c40feb0 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:20.003935 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0bf1e-69f4-4fd0-affd-8f170c40feb0 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:20.004215 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0bf1e-69f4-4fd0-affd-8f170c40feb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:20.049900 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-e3d0bf1e-69f4-4fd0-affd-8f170c40feb0 None None] MFA Rules not processed for user `a50983a75196418eb3f00328a1a60213`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:20.051654 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e3d0bf1e-69f4-4fd0-affd-8f170c40feb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:20.071095 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 818/1623] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:20.077534 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0bf1e-69f4-4fd0-affd-8f170c40feb0 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:20.105163 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0bf1e-69f4-4fd0-affd-8f170c40feb0 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:20.125435 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2622e63-2ca1-4b04-865a-23c4d8753dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:20.126138 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2622e63-2ca1-4b04-865a-23c4d8753dee admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:20.127864 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2622e63-2ca1-4b04-865a-23c4d8753dee admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:20.128161 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2622e63-2ca1-4b04-865a-23c4d8753dee admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:20.128941 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2622e63-2ca1-4b04-865a-23c4d8753dee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:20.130576 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2622e63-2ca1-4b04-865a-23c4d8753dee admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:20.159621 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda10710-4bc4-4dcb-94ba-4de79099633e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:20.161298 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda10710-4bc4-4dcb-94ba-4de79099633e service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:20.172951 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 819/1624] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:20 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:58:20.174778 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817fe81-23a8-49c3-b59f-f34ad0e12ac8 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:20.175734 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817fe81-23a8-49c3-b59f-f34ad0e12ac8 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:20.176671 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817fe81-23a8-49c3-b59f-f34ad0e12ac8 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:20.179233 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 820/1625] 10.222.0.23 () {54 vars in 960 bytes} [Fri May 6 14:58:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:58:20.180224 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8817fe81-23a8-49c3-b59f-f34ad0e12ac8 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:20.187586 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 806/1626] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:58:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:58:20.197209 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592459d2-ceb2-433f-bc28-bd9cbd34ce61 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:20.197600 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592459d2-ceb2-433f-bc28-bd9cbd34ce61 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:20.197883 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592459d2-ceb2-433f-bc28-bd9cbd34ce61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:20.229408 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8817fe81-23a8-49c3-b59f-f34ad0e12ac8 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:20.243799 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71def328-c9a9-43ae-8b1d-d8a819eda7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:20.245344 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71def328-c9a9-43ae-8b1d-d8a819eda7c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:20.246223 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71def328-c9a9-43ae-8b1d-d8a819eda7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:20.246736 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71def328-c9a9-43ae-8b1d-d8a819eda7c6 admin admin] PATH_INFO: `/v3/projects/f0b842b03a8c4beda117c1710302aae0/users/6336dc5822f34ac7bb02f8aa56d5c9c9/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:20.265864 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-592459d2-ceb2-433f-bc28-bd9cbd34ce61 None None] MFA Rules not processed for user `fc8cf09642ec4e10a80badcf8b585ef8`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:20.267094 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-592459d2-ceb2-433f-bc28-bd9cbd34ce61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:20.284375 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71def328-c9a9-43ae-8b1d-d8a819eda7c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0b842b03a8c4beda117c1710302aae0, user_id=6336dc5822f34ac7bb02f8aa56d5c9c9, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:20.294016 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71def328-c9a9-43ae-8b1d-d8a819eda7c6 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:20.307550 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 807/1627] 10.222.0.23 () {58 vars in 1057 bytes} [Fri May 6 14:58:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:20.317416 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592459d2-ceb2-433f-bc28-bd9cbd34ce61 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:20.332001 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 821/1628] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:58:20 2022] PUT /identity/v3/projects/f0b842b03a8c4beda117c1710302aae0/users/6336dc5822f34ac7bb02f8aa56d5c9c9/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:20.335966 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71def328-c9a9-43ae-8b1d-d8a819eda7c6 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:20.340198 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592459d2-ceb2-433f-bc28-bd9cbd34ce61 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:20.491190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71def328-c9a9-43ae-8b1d-d8a819eda7c6 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:20.506085 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353627bc-1b26-4721-b111-56ae8aa4b03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:20.508967 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353627bc-1b26-4721-b111-56ae8aa4b03b admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:20.510138 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353627bc-1b26-4721-b111-56ae8aa4b03b admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:20.510763 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353627bc-1b26-4721-b111-56ae8aa4b03b admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:20.513407 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353627bc-1b26-4721-b111-56ae8aa4b03b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:20.515781 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b9ca1f-49b9-4e89-aa01-de40f15bdfbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:20.516458 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9ca1f-49b9-4e89-aa01-de40f15bdfbf service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:20.517530 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353627bc-1b26-4721-b111-56ae8aa4b03b admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:20.519390 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9ca1f-49b9-4e89-aa01-de40f15bdfbf service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:20.519717 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9ca1f-49b9-4e89-aa01-de40f15bdfbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:20.553513 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 822/1629] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:20 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 06 14:58:20.556596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353627bc-1b26-4721-b111-56ae8aa4b03b admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:20.583091 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353627bc-1b26-4721-b111-56ae8aa4b03b admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:20.599393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3d3cd5-5286-4342-b701-3120266f97b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:20.600303 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d3cd5-5286-4342-b701-3120266f97b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:20.600591 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d3cd5-5286-4342-b701-3120266f97b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:20.601038 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d3cd5-5286-4342-b701-3120266f97b9 admin admin] PATH_INFO: `/v3/projects/f0b842b03a8c4beda117c1710302aae0/users/6336dc5822f34ac7bb02f8aa56d5c9c9/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:20.605878 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3d3cd5-5286-4342-b701-3120266f97b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0b842b03a8c4beda117c1710302aae0, user_id=6336dc5822f34ac7bb02f8aa56d5c9c9, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:20.610428 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3d3cd5-5286-4342-b701-3120266f97b9 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:20.650403 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 823/1630] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:58:20 2022] PUT /identity/v3/projects/f0b842b03a8c4beda117c1710302aae0/users/6336dc5822f34ac7bb02f8aa56d5c9c9/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:20.652860 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f8727-ed52-4cd6-8f75-4432332acc88 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:20.656708 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f8727-ed52-4cd6-8f75-4432332acc88 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:20.658629 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f8727-ed52-4cd6-8f75-4432332acc88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:20.668454 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b9ca1f-49b9-4e89-aa01-de40f15bdfbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:20.670484 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b9ca1f-49b9-4e89-aa01-de40f15bdfbf service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:20.707800 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-f43f8727-ed52-4cd6-8f75-4432332acc88 None None] MFA Rules not processed for user `6336dc5822f34ac7bb02f8aa56d5c9c9`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:20.710973 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 808/1631] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:58:20 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 14:58:20.712101 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f43f8727-ed52-4cd6-8f75-4432332acc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:20.914290 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 824/1632] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:20 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:20.933330 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b9ca1f-49b9-4e89-aa01-de40f15bdfbf service nova] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:20.971178 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf976432-e2c8-49ea-9490-b1b16529e7a3 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:20.971178 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf976432-e2c8-49ea-9490-b1b16529e7a3 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:20.971511 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf976432-e2c8-49ea-9490-b1b16529e7a3 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:20.973473 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 825/1633] 10.222.0.23 () {54 vars in 912 bytes} [Fri May 6 14:58:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:58:21.000310 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aec499-4d16-4e4d-b775-7ac4ed816ffa None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:21.000643 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aec499-4d16-4e4d-b775-7ac4ed816ffa None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:21.001175 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aec499-4d16-4e4d-b775-7ac4ed816ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:21.050099 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-18aec499-4d16-4e4d-b775-7ac4ed816ffa None None] MFA Rules not processed for user `fc8cf09642ec4e10a80badcf8b585ef8`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:21.055344 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-18aec499-4d16-4e4d-b775-7ac4ed816ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:21.123950 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b9ca1f-49b9-4e89-aa01-de40f15bdfbf service nova] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:21.271478 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ca74e1-723e-40ed-8195-084e0340dc3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:21.272777 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca74e1-723e-40ed-8195-084e0340dc3a service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:21.273346 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca74e1-723e-40ed-8195-084e0340dc3a service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:21.274151 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca74e1-723e-40ed-8195-084e0340dc3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:21.289136 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ca74e1-723e-40ed-8195-084e0340dc3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:21.291034 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ca74e1-723e-40ed-8195-084e0340dc3a service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:21.310381 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 826/1634] 10.222.0.23 () {58 vars in 1009 bytes} [Fri May 6 14:58:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:21.310970 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 809/1635] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:58:20 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:58:21.311952 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded23bf-49da-485d-a098-750e91296339 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:21.312244 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded23bf-49da-485d-a098-750e91296339 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:21.312521 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded23bf-49da-485d-a098-750e91296339 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:21.314303 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 827/1636] 10.222.0.23 () {54 vars in 912 bytes} [Fri May 6 14:58:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:58:21.320382 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3479d-4096-46bb-b4ec-853c99d48a31 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:21.320724 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3479d-4096-46bb-b4ec-853c99d48a31 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:21.321003 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3479d-4096-46bb-b4ec-853c99d48a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:21.325551 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cded23bf-49da-485d-a098-750e91296339 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:21.337086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cded23bf-49da-485d-a098-750e91296339 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:21.341652 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7461bd21-4dcf-461a-9466-c9a6162d62c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:21.342743 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461bd21-4dcf-461a-9466-c9a6162d62c8 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:21.343277 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461bd21-4dcf-461a-9466-c9a6162d62c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:21.343791 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461bd21-4dcf-461a-9466-c9a6162d62c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:21.350001 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-fba3479d-4096-46bb-b4ec-853c99d48a31 None None] MFA Rules not processed for user `fc8cf09642ec4e10a80badcf8b585ef8`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:21.353467 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-fba3479d-4096-46bb-b4ec-853c99d48a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:21.364970 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7461bd21-4dcf-461a-9466-c9a6162d62c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:21.371254 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7461bd21-4dcf-461a-9466-c9a6162d62c8 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:21.375578 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 810/1637] 10.222.0.23 () {58 vars in 1009 bytes} [Fri May 6 14:58:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:21.388894 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba3479d-4096-46bb-b4ec-853c99d48a31 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:21.388894 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-fba3479d-4096-46bb-b4ec-853c99d48a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:21.404036 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 828/1638] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:58:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 14:58:21.488117 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7461bd21-4dcf-461a-9466-c9a6162d62c8 service cinder] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:21.498154 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba3479d-4096-46bb-b4ec-853c99d48a31 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:21.502022 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb67323b-990a-4576-ad01-d932615d16ca service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9be93f9ff7e46babc8ee87b50b9c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:21.502681 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67323b-990a-4576-ad01-d932615d16ca service glance] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:21.502959 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67323b-990a-4576-ad01-d932615d16ca service glance] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:21.503230 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67323b-990a-4576-ad01-d932615d16ca service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:21.509945 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7461bd21-4dcf-461a-9466-c9a6162d62c8 service cinder] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:21.516389 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb67323b-990a-4576-ad01-d932615d16ca service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:21.518161 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb67323b-990a-4576-ad01-d932615d16ca service glance] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:21.524064 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a108b4cb-730d-4edb-b442-47571d861c19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:21.525914 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a108b4cb-730d-4edb-b442-47571d861c19 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:21.530688 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a108b4cb-730d-4edb-b442-47571d861c19 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:21.531653 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a108b4cb-730d-4edb-b442-47571d861c19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:21.555083 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 811/1639] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:58:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:58:21.598298 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb67323b-990a-4576-ad01-d932615d16ca service glance] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:21.623752 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a108b4cb-730d-4edb-b442-47571d861c19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:21.625766 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a108b4cb-730d-4edb-b442-47571d861c19 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:21.639238 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 829/1640] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:58:21 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 14:58:21.738113 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb67323b-990a-4576-ad01-d932615d16ca service glance] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:21.745629 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67e6fd-9636-4beb-b313-3f16ce636384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:21.747115 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67e6fd-9636-4beb-b313-3f16ce636384 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:21.747472 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67e6fd-9636-4beb-b313-3f16ce636384 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:21.747850 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67e6fd-9636-4beb-b313-3f16ce636384 admin admin] PATH_INFO: `/v3/users/bd41a5b586e04553bc0fa64c2a8b1ba1` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:21.749565 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b67e6fd-9636-4beb-b313-3f16ce636384 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd41a5b586e04553bc0fa64c2a8b1ba1)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:21.754050 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b67e6fd-9636-4beb-b313-3f16ce636384 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:21.859367 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-2b67e6fd-9636-4beb-b313-3f16ce636384 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd41a5b586e04553bc0fa64c2a8b1ba1'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:21.860706 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-2b67e6fd-9636-4beb-b313-3f16ce636384 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd41a5b586e04553bc0fa64c2a8b1ba1'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:21.872159 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-2b67e6fd-9636-4beb-b313-3f16ce636384 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd41a5b586e04553bc0fa64c2a8b1ba1'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:21.882211 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-2b67e6fd-9636-4beb-b313-3f16ce636384 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd41a5b586e04553bc0fa64c2a8b1ba1'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:21.883532 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-2b67e6fd-9636-4beb-b313-3f16ce636384 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd41a5b586e04553bc0fa64c2a8b1ba1'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:21.916416 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 812/1641] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 14:58:21 2022] DELETE /identity/v3/users/bd41a5b586e04553bc0fa64c2a8b1ba1 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:21.918261 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da358d8-805f-47c6-bf65-647ed5c40dee None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:21.918768 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da358d8-805f-47c6-bf65-647ed5c40dee None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:21.918889 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da358d8-805f-47c6-bf65-647ed5c40dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:21.938584 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-9da358d8-805f-47c6-bf65-647ed5c40dee None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:21.939856 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-9da358d8-805f-47c6-bf65-647ed5c40dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:22.171355 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 830/1642] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:22.239846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67e6fd-9636-4beb-b313-3f16ce636384 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:22.240483 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-2b67e6fd-9636-4beb-b313-3f16ce636384 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:22.385536 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67e6fd-9636-4beb-b313-3f16ce636384 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:22.392452 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b217a6-a459-4a97-bada-8859ee715d72 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:22.392829 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b217a6-a459-4a97-bada-8859ee715d72 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:22.393104 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b217a6-a459-4a97-bada-8859ee715d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:22.394279 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca07a30f-77c4-4c49-a012-9a91a1fd2778 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:22.395479 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca07a30f-77c4-4c49-a012-9a91a1fd2778 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:22.395996 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca07a30f-77c4-4c49-a012-9a91a1fd2778 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:22.396481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca07a30f-77c4-4c49-a012-9a91a1fd2778 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:22.409073 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca07a30f-77c4-4c49-a012-9a91a1fd2778 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:22.410942 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca07a30f-77c4-4c49-a012-9a91a1fd2778 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:22.430954 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 813/1643] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:58:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:58:22.435559 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-c9b217a6-a459-4a97-bada-8859ee715d72 None None] MFA Rules not processed for user `684316ea5603441a93c5390572f7fbd0`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:22.437301 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-c9b217a6-a459-4a97-bada-8859ee715d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:22.569305 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6859b-8b0c-4438-943c-7ab7ddf5d064 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:22.570762 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6859b-8b0c-4438-943c-7ab7ddf5d064 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:22.571303 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6859b-8b0c-4438-943c-7ab7ddf5d064 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:22.574021 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 814/1644] 10.222.0.23 () {54 vars in 912 bytes} [Fri May 6 14:58:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 06 14:58:22.580436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f44c862-5ec3-4d16-a236-12bc0bbea7cf None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:22.580837 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f44c862-5ec3-4d16-a236-12bc0bbea7cf None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:22.581256 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f44c862-5ec3-4d16-a236-12bc0bbea7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:22.610156 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 831/1645] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:22 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:22.611772 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b217a6-a459-4a97-bada-8859ee715d72 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:22.616123 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-c9b217a6-a459-4a97-bada-8859ee715d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:22.651162 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b217a6-a459-4a97-bada-8859ee715d72 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:22.665938 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ea626-4e35-4f04-90ac-45e9e43fe5e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:22.666798 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ea626-4e35-4f04-90ac-45e9e43fe5e5 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:22.667191 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ea626-4e35-4f04-90ac-45e9e43fe5e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:22.667470 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ea626-4e35-4f04-90ac-45e9e43fe5e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:22.668215 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6ea626-4e35-4f04-90ac-45e9e43fe5e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:22.671983 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6ea626-4e35-4f04-90ac-45e9e43fe5e5 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:22.687361 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-7f44c862-5ec3-4d16-a236-12bc0bbea7cf None None] MFA Rules not processed for user `4dcd1542d8f643c4996142a88b9b6abb`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:22.692201 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-7f44c862-5ec3-4d16-a236-12bc0bbea7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:22.720462 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 815/1646] 10.222.0.23 () {58 vars in 1009 bytes} [Fri May 6 14:58:22 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:22.743918 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 832/1647] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:58:22 2022] POST /identity/v3/projects => generated 401 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:58:22.746383 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f44c862-5ec3-4d16-a236-12bc0bbea7cf None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:22.752547 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ea626-4e35-4f04-90ac-45e9e43fe5e5 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:22.754483 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-de6ea626-4e35-4f04-90ac-45e9e43fe5e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:22.779774 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f44c862-5ec3-4d16-a236-12bc0bbea7cf None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:22.883683 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ea626-4e35-4f04-90ac-45e9e43fe5e5 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:22.906588 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65138be-df94-47a0-b8e9-a0b4e2042a97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:22.909069 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65138be-df94-47a0-b8e9-a0b4e2042a97 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:22.909439 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65138be-df94-47a0-b8e9-a0b4e2042a97 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:22.909810 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65138be-df94-47a0-b8e9-a0b4e2042a97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:22.932045 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65138be-df94-47a0-b8e9-a0b4e2042a97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:22.935911 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65138be-df94-47a0-b8e9-a0b4e2042a97 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:22.954203 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 833/1648] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:58:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:58:22.973521 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ee7b2e-b917-48bc-9b66-1b62dae1e208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:22.974229 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee7b2e-b917-48bc-9b66-1b62dae1e208 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:22.974628 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee7b2e-b917-48bc-9b66-1b62dae1e208 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:22.974828 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee7b2e-b917-48bc-9b66-1b62dae1e208 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:22.978256 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ee7b2e-b917-48bc-9b66-1b62dae1e208 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:22.979470 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65138be-df94-47a0-b8e9-a0b4e2042a97 service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:22.981816 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-c65138be-df94-47a0-b8e9-a0b4e2042a97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:23.000918 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ee7b2e-b917-48bc-9b66-1b62dae1e208 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:23.035804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65138be-df94-47a0-b8e9-a0b4e2042a97 service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:23.050097 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb46100-8400-421e-acfe-ec8fc16fbac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:23.062627 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb46100-8400-421e-acfe-ec8fc16fbac2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:23.062991 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb46100-8400-421e-acfe-ec8fc16fbac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:23.064949 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb46100-8400-421e-acfe-ec8fc16fbac2 admin admin] PATH_INFO: `/v3/projects/09c7b905c6444fa3949c574f6084fead` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:23.066548 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb46100-8400-421e-acfe-ec8fc16fbac2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09c7b905c6444fa3949c574f6084fead)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:23.068538 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb46100-8400-421e-acfe-ec8fc16fbac2 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:23.072239 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 816/1649] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:58:22 2022] POST /identity/v3/users => generated 307 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:58:23.078401 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ee7b2e-b917-48bc-9b66-1b62dae1e208 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:23.090729 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ee7b2e-b917-48bc-9b66-1b62dae1e208 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:23.097216 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782cb08b-f7b3-45ff-b750-8169f2d60206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:23.098617 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782cb08b-f7b3-45ff-b750-8169f2d60206 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:23.099059 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782cb08b-f7b3-45ff-b750-8169f2d60206 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:23.100341 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782cb08b-f7b3-45ff-b750-8169f2d60206 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:23.101274 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782cb08b-f7b3-45ff-b750-8169f2d60206 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:23.103649 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782cb08b-f7b3-45ff-b750-8169f2d60206 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:23.127708 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 817/1650] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 06 14:58:23.143019 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782cb08b-f7b3-45ff-b750-8169f2d60206 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:23.161787 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782cb08b-f7b3-45ff-b750-8169f2d60206 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:23.171946 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-4fb46100-8400-421e-acfe-ec8fc16fbac2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09c7b905c6444fa3949c574f6084fead', 'request_id': 'req-4fb46100-8400-421e-acfe-ec8fc16fbac2'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:23.189200 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-4fb46100-8400-421e-acfe-ec8fc16fbac2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09c7b905c6444fa3949c574f6084fead', 'request_id': 'req-4fb46100-8400-421e-acfe-ec8fc16fbac2'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:23.206339 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-4fb46100-8400-421e-acfe-ec8fc16fbac2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09c7b905c6444fa3949c574f6084fead', 'request_id': 'req-4fb46100-8400-421e-acfe-ec8fc16fbac2'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:23.235331 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-4fb46100-8400-421e-acfe-ec8fc16fbac2 admin admin] The token cache is being invalidate because project 09c7b905c6444fa3949c574f6084fead was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 14:58:23.235775 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-4fb46100-8400-421e-acfe-ec8fc16fbac2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:23.237080 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-4fb46100-8400-421e-acfe-ec8fc16fbac2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:23.239751 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 834/1651] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 14:58:22 2022] DELETE /identity/v3/projects/09c7b905c6444fa3949c574f6084fead => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 14:58:23.250921 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3589874-cd6e-4209-a2bc-17086d75e93f None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:23.251233 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3589874-cd6e-4209-a2bc-17086d75e93f None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:23.251515 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3589874-cd6e-4209-a2bc-17086d75e93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:23.293124 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb878e-05e1-47e7-a95e-e7154fdaedea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:23.294649 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb878e-05e1-47e7-a95e-e7154fdaedea admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:23.295307 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb878e-05e1-47e7-a95e-e7154fdaedea admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:23.295600 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb878e-05e1-47e7-a95e-e7154fdaedea admin admin] PATH_INFO: `/v3/projects/a400d4befefc422c881ab8d3900a3d7e/users/e73ad181e1ca4d748dfe06ebc38a7927/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:23.346609 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-b3589874-cd6e-4209-a2bc-17086d75e93f None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:23.352273 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-b3589874-cd6e-4209-a2bc-17086d75e93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:23.371901 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bb878e-05e1-47e7-a95e-e7154fdaedea admin admin] RBAC: Authorizing `identity:create_grant(project_id=a400d4befefc422c881ab8d3900a3d7e, user_id=e73ad181e1ca4d748dfe06ebc38a7927, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:23.374951 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bb878e-05e1-47e7-a95e-e7154fdaedea admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:23.387572 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 835/1652] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:23.396495 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e30734-a5ba-4afd-855e-93ef0c3833c7 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:23.397065 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e30734-a5ba-4afd-855e-93ef0c3833c7 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:23.397507 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e30734-a5ba-4afd-855e-93ef0c3833c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:23.406236 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 818/1653] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:58:23 2022] PUT /identity/v3/projects/a400d4befefc422c881ab8d3900a3d7e/users/e73ad181e1ca4d748dfe06ebc38a7927/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:23.409300 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb878e-05e1-47e7-a95e-e7154fdaedea admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:23.411419 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-27bb878e-05e1-47e7-a95e-e7154fdaedea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:23.454809 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-a0e30734-a5ba-4afd-855e-93ef0c3833c7 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:23.456133 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-a0e30734-a5ba-4afd-855e-93ef0c3833c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:23.582593 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb878e-05e1-47e7-a95e-e7154fdaedea admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:23.595730 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d676154b-571c-44b3-95d5-f5cfa376d25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:23.595999 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676154b-571c-44b3-95d5-f5cfa376d25c admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:23.596204 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676154b-571c-44b3-95d5-f5cfa376d25c admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:23.596507 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676154b-571c-44b3-95d5-f5cfa376d25c admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:23.597260 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d676154b-571c-44b3-95d5-f5cfa376d25c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:23.598452 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 836/1654] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:23.601701 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d676154b-571c-44b3-95d5-f5cfa376d25c admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:23.602649 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e30734-a5ba-4afd-855e-93ef0c3833c7 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:23.627159 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 819/1655] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:58:23.627506 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e30734-a5ba-4afd-855e-93ef0c3833c7 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:23.629622 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d676154b-571c-44b3-95d5-f5cfa376d25c admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:23.642345 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7cf62a-2291-4ae4-963c-9fe9278d861f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:23.643032 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7cf62a-2291-4ae4-963c-9fe9278d861f admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:23.643320 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7cf62a-2291-4ae4-963c-9fe9278d861f admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:23.643598 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7cf62a-2291-4ae4-963c-9fe9278d861f admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:23.644320 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7cf62a-2291-4ae4-963c-9fe9278d861f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:58:23.644664 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7cf62a-2291-4ae4-963c-9fe9278d861f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:23.646291 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7cf62a-2291-4ae4-963c-9fe9278d861f admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:23.668129 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d676154b-571c-44b3-95d5-f5cfa376d25c admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:23.681724 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 837/1656] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 14:58:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:58:23.686948 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7cf62a-2291-4ae4-963c-9fe9278d861f admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:23.704287 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca225c-fa4c-468b-93d2-95ac4fea6893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:23.705253 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca225c-fa4c-468b-93d2-95ac4fea6893 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:23.705543 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca225c-fa4c-468b-93d2-95ac4fea6893 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:23.705819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca225c-fa4c-468b-93d2-95ac4fea6893 admin admin] PATH_INFO: `/v3/projects/a400d4befefc422c881ab8d3900a3d7e/users/e73ad181e1ca4d748dfe06ebc38a7927/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:23.708201 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca225c-fa4c-468b-93d2-95ac4fea6893 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a400d4befefc422c881ab8d3900a3d7e, user_id=e73ad181e1ca4d748dfe06ebc38a7927, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:23.713301 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca225c-fa4c-468b-93d2-95ac4fea6893 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:23.758891 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 820/1657] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:58:23 2022] PUT /identity/v3/projects/a400d4befefc422c881ab8d3900a3d7e/users/e73ad181e1ca4d748dfe06ebc38a7927/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:23.776225 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7cf62a-2291-4ae4-963c-9fe9278d861f admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:23.962447 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c6960-8fff-4bf0-a304-7d9579d6c315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:23.965503 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c6960-8fff-4bf0-a304-7d9579d6c315 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:23.966154 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c6960-8fff-4bf0-a304-7d9579d6c315 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:23.966970 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c6960-8fff-4bf0-a304-7d9579d6c315 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:23.973429 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602c6960-8fff-4bf0-a304-7d9579d6c315 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:23.976890 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602c6960-8fff-4bf0-a304-7d9579d6c315 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:24.055715 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 838/1658] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:58:23 2022] POST /identity/v3/projects => generated 357 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 14:58:24.058261 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca225c-fa4c-468b-93d2-95ac4fea6893 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:24.079434 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca225c-fa4c-468b-93d2-95ac4fea6893 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:24.212085 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd9452-ac67-42c4-b483-cc61e1a03a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:24.212838 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd9452-ac67-42c4-b483-cc61e1a03a44 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:24.214544 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd9452-ac67-42c4-b483-cc61e1a03a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:24.214544 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd9452-ac67-42c4-b483-cc61e1a03a44 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:24.214785 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bd9452-ac67-42c4-b483-cc61e1a03a44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:24.217012 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bd9452-ac67-42c4-b483-cc61e1a03a44 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:24.301725 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 821/1659] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:58:24 2022] POST /identity/v3/users => generated 279 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:58:24.305711 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c6960-8fff-4bf0-a304-7d9579d6c315 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:24.332842 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c6960-8fff-4bf0-a304-7d9579d6c315 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:24.341272 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d1d3d-edc7-4442-97d2-deab96b2fadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:24.343183 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d1d3d-edc7-4442-97d2-deab96b2fadb admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:24.343613 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d1d3d-edc7-4442-97d2-deab96b2fadb admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:24.344666 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d1d3d-edc7-4442-97d2-deab96b2fadb admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:24.349580 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6d1d3d-edc7-4442-97d2-deab96b2fadb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:24.352904 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6d1d3d-edc7-4442-97d2-deab96b2fadb admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:24.373349 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 839/1660] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:24 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 06 14:58:24.377320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd9452-ac67-42c4-b483-cc61e1a03a44 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:24.394925 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd9452-ac67-42c4-b483-cc61e1a03a44 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:24.402047 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b03ca-e6fb-4107-bd12-a6e6ff7b2c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:24.402047 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b03ca-e6fb-4107-bd12-a6e6ff7b2c08 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:24.402792 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b03ca-e6fb-4107-bd12-a6e6ff7b2c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:24.403637 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b03ca-e6fb-4107-bd12-a6e6ff7b2c08 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:24.406337 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85b03ca-e6fb-4107-bd12-a6e6ff7b2c08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:24.408538 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85b03ca-e6fb-4107-bd12-a6e6ff7b2c08 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:24.437441 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 822/1661] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:24 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:58:24.442466 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d1d3d-edc7-4442-97d2-deab96b2fadb admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:24.484438 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d1d3d-edc7-4442-97d2-deab96b2fadb admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:24.504562 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2992c1-408e-4949-863a-ee1f91b75b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:24.505703 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2992c1-408e-4949-863a-ee1f91b75b16 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:24.506107 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2992c1-408e-4949-863a-ee1f91b75b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:24.506496 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2992c1-408e-4949-863a-ee1f91b75b16 admin admin] PATH_INFO: `/v3/projects/1b8d2d8b71924fe8a56edf41750ca99b/users/025d805b26d84e3994a9d17660372016/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:24.573671 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2992c1-408e-4949-863a-ee1f91b75b16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b8d2d8b71924fe8a56edf41750ca99b, user_id=025d805b26d84e3994a9d17660372016, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:24.588065 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2992c1-408e-4949-863a-ee1f91b75b16 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:24.661971 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 840/1662] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:58:24 2022] PUT /identity/v3/projects/1b8d2d8b71924fe8a56edf41750ca99b/users/025d805b26d84e3994a9d17660372016/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:24.677207 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e3c28-e4f3-49ec-8210-e6812eb56e3b None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:24.677558 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e3c28-e4f3-49ec-8210-e6812eb56e3b None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:24.677864 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e3c28-e4f3-49ec-8210-e6812eb56e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:24.701294 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59086b08-bde2-444b-9358-ac73643a9767 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:24.701568 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59086b08-bde2-444b-9358-ac73643a9767 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:24.701879 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59086b08-bde2-444b-9358-ac73643a9767 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:24.703480 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 841/1663] 10.222.0.23 () {54 vars in 960 bytes} [Fri May 6 14:58:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:58:24.711819 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93bf760-445c-4ccf-a452-831d33d72732 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:24.712233 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93bf760-445c-4ccf-a452-831d33d72732 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:24.712525 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93bf760-445c-4ccf-a452-831d33d72732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:24.756812 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-e93bf760-445c-4ccf-a452-831d33d72732 None None] MFA Rules not processed for user `887a619ef5b5417db98ff15a9f712724`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:24.759309 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e93bf760-445c-4ccf-a452-831d33d72732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:24.776895 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-3f4e3c28-e4f3-49ec-8210-e6812eb56e3b None None] MFA Rules not processed for user `025d805b26d84e3994a9d17660372016`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:24.782538 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-3f4e3c28-e4f3-49ec-8210-e6812eb56e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:24.984271 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 842/1664] 10.222.0.23 () {58 vars in 1057 bytes} [Fri May 6 14:58:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 14:58:24.996370 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93bf760-445c-4ccf-a452-831d33d72732 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:24.999022 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 823/1665] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:24 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:25.009125 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e3c28-e4f3-49ec-8210-e6812eb56e3b None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:25.023801 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93bf760-445c-4ccf-a452-831d33d72732 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:25.047289 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae6997-4f68-468c-b136-7b4d6747f239 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887a619ef5b5417db98ff15a9f712724', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:25.047964 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae6997-4f68-468c-b136-7b4d6747f239 service placement] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:25.048251 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae6997-4f68-468c-b136-7b4d6747f239 service placement] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:25.048535 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae6997-4f68-468c-b136-7b4d6747f239 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:25.055406 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-8aae6997-4f68-468c-b136-7b4d6747f239 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:25.110226 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae6997-4f68-468c-b136-7b4d6747f239 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:25.112243 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae6997-4f68-468c-b136-7b4d6747f239 service placement] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:25.156700 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 843/1666] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:58:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 14:58:25.243483 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e3c28-e4f3-49ec-8210-e6812eb56e3b None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:25.269456 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2514887c-f218-4ec6-9923-cfced922179d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:25.270401 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2514887c-f218-4ec6-9923-cfced922179d admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:25.270759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2514887c-f218-4ec6-9923-cfced922179d admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:25.270979 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2514887c-f218-4ec6-9923-cfced922179d admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:25.271785 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2514887c-f218-4ec6-9923-cfced922179d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:25.273672 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2514887c-f218-4ec6-9923-cfced922179d admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:25.345580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 824/1667] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:58:25 2022] POST /identity/v3/projects => generated 357 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:58:25.347887 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae6997-4f68-468c-b136-7b4d6747f239 service placement] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:25.362005 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae6997-4f68-468c-b136-7b4d6747f239 service placement] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:25.561463 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d50ed8-21ff-4258-96bf-ad19d0941832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:25.562112 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d50ed8-21ff-4258-96bf-ad19d0941832 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:25.562389 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d50ed8-21ff-4258-96bf-ad19d0941832 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:25.562681 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d50ed8-21ff-4258-96bf-ad19d0941832 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:25.563685 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d50ed8-21ff-4258-96bf-ad19d0941832 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:25.565845 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d50ed8-21ff-4258-96bf-ad19d0941832 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:25.622995 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 844/1668] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:58:25 2022] POST /identity/v3/users => generated 285 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:58:25.624890 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2514887c-f218-4ec6-9923-cfced922179d admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:25.647619 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2514887c-f218-4ec6-9923-cfced922179d admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:25.660943 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74149ccd-1125-4c31-b059-69ec2d057737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:25.661636 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74149ccd-1125-4c31-b059-69ec2d057737 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:25.661923 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74149ccd-1125-4c31-b059-69ec2d057737 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:25.662200 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74149ccd-1125-4c31-b059-69ec2d057737 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:25.665166 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74149ccd-1125-4c31-b059-69ec2d057737 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:25.669821 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74149ccd-1125-4c31-b059-69ec2d057737 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:25.702850 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 825/1669] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:58:25.704197 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d50ed8-21ff-4258-96bf-ad19d0941832 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:25.749023 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d50ed8-21ff-4258-96bf-ad19d0941832 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:25.769173 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f629620e-a3c3-4b15-a5b7-ee6f08ba3fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:25.770135 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629620e-a3c3-4b15-a5b7-ee6f08ba3fd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:25.770424 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629620e-a3c3-4b15-a5b7-ee6f08ba3fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:25.771261 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629620e-a3c3-4b15-a5b7-ee6f08ba3fd1 admin admin] PATH_INFO: `/v3/projects/9975f78865eb4440a2587efc16fa7283/users/bf31486468904557ae04cdbf7de06b70/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:25.834601 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f629620e-a3c3-4b15-a5b7-ee6f08ba3fd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9975f78865eb4440a2587efc16fa7283, user_id=bf31486468904557ae04cdbf7de06b70, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:25.843439 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f629620e-a3c3-4b15-a5b7-ee6f08ba3fd1 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:25.874052 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 845/1670] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:58:25 2022] PUT /identity/v3/projects/9975f78865eb4440a2587efc16fa7283/users/bf31486468904557ae04cdbf7de06b70/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:25.875382 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74149ccd-1125-4c31-b059-69ec2d057737 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:25.896148 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74149ccd-1125-4c31-b059-69ec2d057737 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:26.042789 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab5f93-e404-4273-bd6e-b1b489699a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:26.043486 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab5f93-e404-4273-bd6e-b1b489699a54 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:26.043768 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab5f93-e404-4273-bd6e-b1b489699a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:26.044052 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab5f93-e404-4273-bd6e-b1b489699a54 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:26.044805 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbab5f93-e404-4273-bd6e-b1b489699a54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:26.046649 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbab5f93-e404-4273-bd6e-b1b489699a54 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:26.086053 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 826/1671] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:58:26.091368 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f629620e-a3c3-4b15-a5b7-ee6f08ba3fd1 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:26.109219 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f629620e-a3c3-4b15-a5b7-ee6f08ba3fd1 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:26.120242 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b951a-b8de-4d87-948c-a480e3391fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:26.121577 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b951a-b8de-4d87-948c-a480e3391fb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:26.122059 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b951a-b8de-4d87-948c-a480e3391fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:26.123263 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b951a-b8de-4d87-948c-a480e3391fb0 admin admin] PATH_INFO: `/v3/projects/9975f78865eb4440a2587efc16fa7283/users/bf31486468904557ae04cdbf7de06b70/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:26.136914 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5b951a-b8de-4d87-948c-a480e3391fb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9975f78865eb4440a2587efc16fa7283, user_id=bf31486468904557ae04cdbf7de06b70, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:26.144339 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5b951a-b8de-4d87-948c-a480e3391fb0 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:26.219371 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 846/1672] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:58:26 2022] PUT /identity/v3/projects/9975f78865eb4440a2587efc16fa7283/users/bf31486468904557ae04cdbf7de06b70/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:26.221143 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f145fe1-6619-4013-9b48-63015e6ae3f7 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:26.221438 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f145fe1-6619-4013-9b48-63015e6ae3f7 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:26.221720 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f145fe1-6619-4013-9b48-63015e6ae3f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:26.322521 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-2f145fe1-6619-4013-9b48-63015e6ae3f7 None None] MFA Rules not processed for user `bf31486468904557ae04cdbf7de06b70`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:26.326026 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-2f145fe1-6619-4013-9b48-63015e6ae3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:26.683486 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 827/1673] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:26 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 464 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:26.701722 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b951a-b8de-4d87-948c-a480e3391fb0 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:26.704011 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-ad5b951a-b8de-4d87-948c-a480e3391fb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:26.863634 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b951a-b8de-4d87-948c-a480e3391fb0 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:26.883369 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f30ea5c-b821-42ae-a87f-95bba4f3dce8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:26.884345 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30ea5c-b821-42ae-a87f-95bba4f3dce8 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:26.885018 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30ea5c-b821-42ae-a87f-95bba4f3dce8 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:26.885698 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30ea5c-b821-42ae-a87f-95bba4f3dce8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:26.904301 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f30ea5c-b821-42ae-a87f-95bba4f3dce8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:26.906426 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f30ea5c-b821-42ae-a87f-95bba4f3dce8 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:26.936368 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 847/1674] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:58:26 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:58:27.111354 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f145fe1-6619-4013-9b48-63015e6ae3f7 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:27.113123 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-2f145fe1-6619-4013-9b48-63015e6ae3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:27.137932 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f145fe1-6619-4013-9b48-63015e6ae3f7 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:27.165239 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fc5da9-2a5e-43f5-9aa3-2863a73da347 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:27.166015 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc5da9-2a5e-43f5-9aa3-2863a73da347 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:27.167204 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc5da9-2a5e-43f5-9aa3-2863a73da347 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:27.167204 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc5da9-2a5e-43f5-9aa3-2863a73da347 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:27.320425 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fc5da9-2a5e-43f5-9aa3-2863a73da347 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:27.323585 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fc5da9-2a5e-43f5-9aa3-2863a73da347 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:27.344402 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 828/1675] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:58:27 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:58:27.699790 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f20d724-ee34-4a08-8b5a-1262e433791b None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:27.701123 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f20d724-ee34-4a08-8b5a-1262e433791b None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:27.701597 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f20d724-ee34-4a08-8b5a-1262e433791b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:27.797147 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-4f20d724-ee34-4a08-8b5a-1262e433791b None None] MFA Rules not processed for user `e73ad181e1ca4d748dfe06ebc38a7927`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:27.798838 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-4f20d724-ee34-4a08-8b5a-1262e433791b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:28.064688 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 848/1676] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:27 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:31.414586 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1ffe6-f1d2-4fcb-b12b-17b76386a767 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:31.417351 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1ffe6-f1d2-4fcb-b12b-17b76386a767 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:31.417653 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1ffe6-f1d2-4fcb-b12b-17b76386a767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:31.487839 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-e5c1ffe6-f1d2-4fcb-b12b-17b76386a767 None None] MFA Rules not processed for user `a7dc1bc53c59423e934f7831ea022c73`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:31.491077 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-e5c1ffe6-f1d2-4fcb-b12b-17b76386a767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:31.543271 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f20d724-ee34-4a08-8b5a-1262e433791b None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:31.720570 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 829/1677] 10.222.0.23 () {58 vars in 1009 bytes} [Fri May 6 14:58:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:31.761305 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f20d724-ee34-4a08-8b5a-1262e433791b None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:31.778763 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce05df-e3a4-43e6-b131-0a6727a9dcaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:31.779960 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce05df-e3a4-43e6-b131-0a6727a9dcaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:31.780533 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce05df-e3a4-43e6-b131-0a6727a9dcaa admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:31.781338 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce05df-e3a4-43e6-b131-0a6727a9dcaa admin admin] PATH_INFO: `/v3/users/684316ea5603441a93c5390572f7fbd0` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:31.784698 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fce05df-e3a4-43e6-b131-0a6727a9dcaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=684316ea5603441a93c5390572f7fbd0)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:31.787682 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: WARNING py.warnings [None req-2fce05df-e3a4-43e6-b131-0a6727a9dcaa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 06 14:58:31.787682 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: warnings.warn(msg) May 06 14:58:31.787682 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: May 06 14:58:31.791924 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fce05df-e3a4-43e6-b131-0a6727a9dcaa admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:31.944167 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-2fce05df-e3a4-43e6-b131-0a6727a9dcaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '684316ea5603441a93c5390572f7fbd0'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:31.967383 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-2fce05df-e3a4-43e6-b131-0a6727a9dcaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '684316ea5603441a93c5390572f7fbd0'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:31.981465 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c1ffe6-f1d2-4fcb-b12b-17b76386a767 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:31.983132 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-e5c1ffe6-f1d2-4fcb-b12b-17b76386a767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:32.005680 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-2fce05df-e3a4-43e6-b131-0a6727a9dcaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '684316ea5603441a93c5390572f7fbd0'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:32.007275 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-2fce05df-e3a4-43e6-b131-0a6727a9dcaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '684316ea5603441a93c5390572f7fbd0'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:32.023624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-2fce05df-e3a4-43e6-b131-0a6727a9dcaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '684316ea5603441a93c5390572f7fbd0'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:32.049081 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 849/1678] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 14:58:31 2022] DELETE /identity/v3/users/684316ea5603441a93c5390572f7fbd0 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 14:58:32.192855 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c1ffe6-f1d2-4fcb-b12b-17b76386a767 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:32.319772 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993c7525-975c-40f3-aa56-e948d8dfbcc9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf9eed62c8f1409282c2af6488426a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:32.320467 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c7525-975c-40f3-aa56-e948d8dfbcc9 service swift] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:32.320786 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c7525-975c-40f3-aa56-e948d8dfbcc9 service swift] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:32.321063 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c7525-975c-40f3-aa56-e948d8dfbcc9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:32.324260 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-993c7525-975c-40f3-aa56-e948d8dfbcc9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:32.435156 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993c7525-975c-40f3-aa56-e948d8dfbcc9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:32.439590 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993c7525-975c-40f3-aa56-e948d8dfbcc9 service swift] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:32.464995 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 830/1679] 10.222.0.23 () {60 vars in 1384 bytes} [Fri May 6 14:58:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 06 14:58:33.685388 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce05df-e3a4-43e6-b131-0a6727a9dcaa admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:33.688339 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-2fce05df-e3a4-43e6-b131-0a6727a9dcaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:33.924582 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce05df-e3a4-43e6-b131-0a6727a9dcaa admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:33.939084 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1f972-3499-4496-b1c2-354def3ab738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:33.941024 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1f972-3499-4496-b1c2-354def3ab738 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:33.941583 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1f972-3499-4496-b1c2-354def3ab738 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:33.942321 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1f972-3499-4496-b1c2-354def3ab738 admin admin] PATH_INFO: `/v3/projects/4637699c99024ca2a25d4afed19102d2` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:33.947014 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a1f972-3499-4496-b1c2-354def3ab738 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4637699c99024ca2a25d4afed19102d2)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:33.950141 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a1f972-3499-4496-b1c2-354def3ab738 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:34.075854 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b9a1f972-3499-4496-b1c2-354def3ab738 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4637699c99024ca2a25d4afed19102d2', 'request_id': 'req-b9a1f972-3499-4496-b1c2-354def3ab738'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:34.095692 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b9a1f972-3499-4496-b1c2-354def3ab738 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4637699c99024ca2a25d4afed19102d2', 'request_id': 'req-b9a1f972-3499-4496-b1c2-354def3ab738'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:34.127995 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b9a1f972-3499-4496-b1c2-354def3ab738 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4637699c99024ca2a25d4afed19102d2', 'request_id': 'req-b9a1f972-3499-4496-b1c2-354def3ab738'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:34.168259 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b9a1f972-3499-4496-b1c2-354def3ab738 admin admin] The token cache is being invalidate because project 4637699c99024ca2a25d4afed19102d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 14:58:34.172077 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b9a1f972-3499-4496-b1c2-354def3ab738 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:34.174803 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b9a1f972-3499-4496-b1c2-354def3ab738 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:34.181164 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 850/1680] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 14:58:33 2022] DELETE /identity/v3/projects/4637699c99024ca2a25d4afed19102d2 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 14:58:34.187246 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993c7525-975c-40f3-aa56-e948d8dfbcc9 service swift] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:34.207295 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993c7525-975c-40f3-aa56-e948d8dfbcc9 service swift] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:34.337495 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1f972-3499-4496-b1c2-354def3ab738 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:34.339255 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-b9a1f972-3499-4496-b1c2-354def3ab738 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:34.491325 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aae038-c6ac-4da9-9a8f-e121db3c65f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:34.495180 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aae038-c6ac-4da9-9a8f-e121db3c65f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:34.495801 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aae038-c6ac-4da9-9a8f-e121db3c65f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:34.496268 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aae038-c6ac-4da9-9a8f-e121db3c65f2 admin admin] PATH_INFO: `/v3/users/e73ad181e1ca4d748dfe06ebc38a7927` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:34.502758 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aae038-c6ac-4da9-9a8f-e121db3c65f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e73ad181e1ca4d748dfe06ebc38a7927)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:34.524023 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aae038-c6ac-4da9-9a8f-e121db3c65f2 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:34.626269 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1f972-3499-4496-b1c2-354def3ab738 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:34.639322 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b253f45c-5d07-4858-9c47-e0b3cb54c8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:34.640367 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b253f45c-5d07-4858-9c47-e0b3cb54c8ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:34.640560 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b253f45c-5d07-4858-9c47-e0b3cb54c8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:34.641068 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b253f45c-5d07-4858-9c47-e0b3cb54c8ed admin admin] PATH_INFO: `/v3/users/27fff5cf15794969b325cedffa2a90e5` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:34.649576 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b253f45c-5d07-4858-9c47-e0b3cb54c8ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=27fff5cf15794969b325cedffa2a90e5)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:34.652111 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b253f45c-5d07-4858-9c47-e0b3cb54c8ed admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:34.691895 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-88aae038-c6ac-4da9-9a8f-e121db3c65f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e73ad181e1ca4d748dfe06ebc38a7927'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:34.693142 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-88aae038-c6ac-4da9-9a8f-e121db3c65f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e73ad181e1ca4d748dfe06ebc38a7927'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:34.720653 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-88aae038-c6ac-4da9-9a8f-e121db3c65f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e73ad181e1ca4d748dfe06ebc38a7927'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:34.735509 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-88aae038-c6ac-4da9-9a8f-e121db3c65f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e73ad181e1ca4d748dfe06ebc38a7927'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:34.737252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-88aae038-c6ac-4da9-9a8f-e121db3c65f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e73ad181e1ca4d748dfe06ebc38a7927'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:34.793179 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 831/1681] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 14:58:34 2022] DELETE /identity/v3/users/e73ad181e1ca4d748dfe06ebc38a7927 => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:34.796572 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b253f45c-5d07-4858-9c47-e0b3cb54c8ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27fff5cf15794969b325cedffa2a90e5'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:34.799337 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237ca54-b9b1-4223-a216-c050df60053f None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:34.803334 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237ca54-b9b1-4223-a216-c050df60053f None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:34.804007 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237ca54-b9b1-4223-a216-c050df60053f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:34.813374 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b253f45c-5d07-4858-9c47-e0b3cb54c8ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27fff5cf15794969b325cedffa2a90e5'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:34.833725 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b253f45c-5d07-4858-9c47-e0b3cb54c8ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27fff5cf15794969b325cedffa2a90e5'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:34.836128 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b253f45c-5d07-4858-9c47-e0b3cb54c8ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27fff5cf15794969b325cedffa2a90e5'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:34.837142 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b253f45c-5d07-4858-9c47-e0b3cb54c8ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27fff5cf15794969b325cedffa2a90e5'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:34.879623 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-5237ca54-b9b1-4223-a216-c050df60053f None None] MFA Rules not processed for user `4dcd1542d8f643c4996142a88b9b6abb`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:34.880267 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 851/1682] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 14:58:34 2022] DELETE /identity/v3/users/27fff5cf15794969b325cedffa2a90e5 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:34.884405 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-5237ca54-b9b1-4223-a216-c050df60053f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:34.896184 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f478dcd-a923-424e-8252-b3cb5120961c None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:34.898613 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f478dcd-a923-424e-8252-b3cb5120961c None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:34.899147 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f478dcd-a923-424e-8252-b3cb5120961c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:34.954906 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-3f478dcd-a923-424e-8252-b3cb5120961c None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:34.960888 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-3f478dcd-a923-424e-8252-b3cb5120961c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:35.114486 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 832/1683] 10.222.0.26 () {58 vars in 1015 bytes} [Fri May 6 14:58:34 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:35.116328 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 852/1684] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:35.128643 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5237ca54-b9b1-4223-a216-c050df60053f None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:35.130239 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-5237ca54-b9b1-4223-a216-c050df60053f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:35.163651 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5237ca54-b9b1-4223-a216-c050df60053f None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:35.169931 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f478dcd-a923-424e-8252-b3cb5120961c None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:35.171202 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f0d4f-c58d-4074-83c1-6425ed261d89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:35.171887 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f0d4f-c58d-4074-83c1-6425ed261d89 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:35.172169 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f0d4f-c58d-4074-83c1-6425ed261d89 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:35.172687 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f0d4f-c58d-4074-83c1-6425ed261d89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:35.196747 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f478dcd-a923-424e-8252-b3cb5120961c None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:35.211945 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3389c1c-bf1d-439c-afe6-88b35ff6264e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:35.214137 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3389c1c-bf1d-439c-afe6-88b35ff6264e service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:35.214660 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3389c1c-bf1d-439c-afe6-88b35ff6264e service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:35.215146 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3389c1c-bf1d-439c-afe6-88b35ff6264e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:35.219670 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3f0d4f-c58d-4074-83c1-6425ed261d89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:35.221574 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3f0d4f-c58d-4074-83c1-6425ed261d89 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:35.226097 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3389c1c-bf1d-439c-afe6-88b35ff6264e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:35.227888 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3389c1c-bf1d-439c-afe6-88b35ff6264e service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:35.243646 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 853/1685] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:58:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:58:35.253758 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 833/1686] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:58:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:58:36.702279 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3389c1c-bf1d-439c-afe6-88b35ff6264e service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:36.707356 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-b3389c1c-bf1d-439c-afe6-88b35ff6264e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:36.746113 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3389c1c-bf1d-439c-afe6-88b35ff6264e service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:36.760880 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b648d8-84dd-469e-ab1d-cb77cc170547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:36.769550 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b648d8-84dd-469e-ab1d-cb77cc170547 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:36.770962 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b648d8-84dd-469e-ab1d-cb77cc170547 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:36.772265 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b648d8-84dd-469e-ab1d-cb77cc170547 admin admin] PATH_INFO: `/v3/projects/30792c0ea24845fdb65dcd2b282d64ec` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:36.777012 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b648d8-84dd-469e-ab1d-cb77cc170547 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30792c0ea24845fdb65dcd2b282d64ec)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:36.785184 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b648d8-84dd-469e-ab1d-cb77cc170547 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:36.955403 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-74b648d8-84dd-469e-ab1d-cb77cc170547 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30792c0ea24845fdb65dcd2b282d64ec', 'request_id': 'req-74b648d8-84dd-469e-ab1d-cb77cc170547'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:36.975366 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-74b648d8-84dd-469e-ab1d-cb77cc170547 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30792c0ea24845fdb65dcd2b282d64ec', 'request_id': 'req-74b648d8-84dd-469e-ab1d-cb77cc170547'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:36.994600 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-74b648d8-84dd-469e-ab1d-cb77cc170547 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30792c0ea24845fdb65dcd2b282d64ec', 'request_id': 'req-74b648d8-84dd-469e-ab1d-cb77cc170547'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:37.040630 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-74b648d8-84dd-469e-ab1d-cb77cc170547 admin admin] The token cache is being invalidate because project 30792c0ea24845fdb65dcd2b282d64ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 14:58:37.041810 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-74b648d8-84dd-469e-ab1d-cb77cc170547 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:37.042649 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-74b648d8-84dd-469e-ab1d-cb77cc170547 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:37.049171 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 854/1687] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 14:58:36 2022] DELETE /identity/v3/projects/30792c0ea24845fdb65dcd2b282d64ec => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:37.050848 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a93e5-0ef2-430c-9570-4a3efa8cc3ab None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:37.051489 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a93e5-0ef2-430c-9570-4a3efa8cc3ab None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:37.051489 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a93e5-0ef2-430c-9570-4a3efa8cc3ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:37.127096 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-d76a93e5-0ef2-430c-9570-4a3efa8cc3ab None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:37.130362 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d76a93e5-0ef2-430c-9570-4a3efa8cc3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:37.329571 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 834/1688] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 14:58:37.339066 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b87f47-4a15-4270-9e7f-b9830f089c37 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:37.339435 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b87f47-4a15-4270-9e7f-b9830f089c37 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:37.339782 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b87f47-4a15-4270-9e7f-b9830f089c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:37.428623 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-80b87f47-4a15-4270-9e7f-b9830f089c37 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:37.429959 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-80b87f47-4a15-4270-9e7f-b9830f089c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:37.466583 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 855/1689] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:37.467954 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76a93e5-0ef2-430c-9570-4a3efa8cc3ab None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:37.483617 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76a93e5-0ef2-430c-9570-4a3efa8cc3ab None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:37.512355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cc3a05-8d76-45ab-a87f-732bfe54d384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:37.513439 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc3a05-8d76-45ab-a87f-732bfe54d384 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:37.514053 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc3a05-8d76-45ab-a87f-732bfe54d384 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:37.514550 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc3a05-8d76-45ab-a87f-732bfe54d384 admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:37.519842 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cc3a05-8d76-45ab-a87f-732bfe54d384 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:58:37.521130 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cc3a05-8d76-45ab-a87f-732bfe54d384 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:37.523110 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cc3a05-8d76-45ab-a87f-732bfe54d384 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:37.594050 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 835/1690] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 14:58:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:58:37.595287 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b87f47-4a15-4270-9e7f-b9830f089c37 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:37.611118 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b87f47-4a15-4270-9e7f-b9830f089c37 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:37.633293 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2878dc29-2e41-4688-8e9f-7472aea9d0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:37.634388 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878dc29-2e41-4688-8e9f-7472aea9d0c9 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:37.634895 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878dc29-2e41-4688-8e9f-7472aea9d0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:37.635357 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878dc29-2e41-4688-8e9f-7472aea9d0c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:37.636769 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2878dc29-2e41-4688-8e9f-7472aea9d0c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:37.639463 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2878dc29-2e41-4688-8e9f-7472aea9d0c9 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:37.743495 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 856/1691] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:58:37 2022] POST /identity/v3/projects => generated 367 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:58:37.750114 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cc3a05-8d76-45ab-a87f-732bfe54d384 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:37.766547 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cddd6e-98f6-4f69-abba-78ed0a56ee7c None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:37.768684 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cddd6e-98f6-4f69-abba-78ed0a56ee7c None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:37.771213 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cddd6e-98f6-4f69-abba-78ed0a56ee7c None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:37.776627 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cc3a05-8d76-45ab-a87f-732bfe54d384 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:37.777888 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 857/1692] 10.222.0.23 () {54 vars in 921 bytes} [Fri May 6 14:58:37 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:58:37.855039 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb2c508-cd78-4ff5-8915-01922fe6af8e None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:37.856199 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb2c508-cd78-4ff5-8915-01922fe6af8e None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:37.856548 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb2c508-cd78-4ff5-8915-01922fe6af8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:37.934002 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-7fb2c508-cd78-4ff5-8915-01922fe6af8e None None] MFA Rules not processed for user `fc8cf09642ec4e10a80badcf8b585ef8`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:37.935289 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-7fb2c508-cd78-4ff5-8915-01922fe6af8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:38.004847 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ec5bf5-2f8b-4b55-8749-377d9a0b3ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:38.005543 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec5bf5-2f8b-4b55-8749-377d9a0b3ec7 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:38.006709 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec5bf5-2f8b-4b55-8749-377d9a0b3ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:38.007329 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec5bf5-2f8b-4b55-8749-377d9a0b3ec7 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:38.008171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ec5bf5-2f8b-4b55-8749-377d9a0b3ec7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:38.010610 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ec5bf5-2f8b-4b55-8749-377d9a0b3ec7 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:38.077374 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 836/1693] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:58:37 2022] POST /identity/v3/users => generated 284 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:58:38.079026 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ec5bf5-2f8b-4b55-8749-377d9a0b3ec7 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:38.102952 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ec5bf5-2f8b-4b55-8749-377d9a0b3ec7 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:38.121394 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265755e2-149c-4ed9-a251-201cb6661cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:38.122371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265755e2-149c-4ed9-a251-201cb6661cee admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:38.122805 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265755e2-149c-4ed9-a251-201cb6661cee admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:38.123229 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265755e2-149c-4ed9-a251-201cb6661cee admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:38.124109 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265755e2-149c-4ed9-a251-201cb6661cee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:38.128257 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265755e2-149c-4ed9-a251-201cb6661cee admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:38.166494 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 837/1694] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:38 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:58:38.168275 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265755e2-149c-4ed9-a251-201cb6661cee admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:38.189382 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265755e2-149c-4ed9-a251-201cb6661cee admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:38.205589 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3347185b-9ce3-447a-97fc-db229a65f252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:38.206600 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3347185b-9ce3-447a-97fc-db229a65f252 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:38.207387 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3347185b-9ce3-447a-97fc-db229a65f252 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:38.207838 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3347185b-9ce3-447a-97fc-db229a65f252 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:38.209490 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3347185b-9ce3-447a-97fc-db229a65f252 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:38.214662 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3347185b-9ce3-447a-97fc-db229a65f252 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:38.251237 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 838/1695] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:38 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:58:38.257826 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3347185b-9ce3-447a-97fc-db229a65f252 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:38.273528 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-3347185b-9ce3-447a-97fc-db229a65f252 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:38.325414 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3347185b-9ce3-447a-97fc-db229a65f252 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:38.349243 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87094215-5cd1-4ac6-bc43-6029b29b246b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:38.350358 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87094215-5cd1-4ac6-bc43-6029b29b246b admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:38.350829 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87094215-5cd1-4ac6-bc43-6029b29b246b admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:38.351246 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87094215-5cd1-4ac6-bc43-6029b29b246b admin admin] PATH_INFO: `/v3/projects/a400d4befefc422c881ab8d3900a3d7e` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:38.354236 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87094215-5cd1-4ac6-bc43-6029b29b246b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a400d4befefc422c881ab8d3900a3d7e)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:38.356795 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: WARNING py.warnings [None req-87094215-5cd1-4ac6-bc43-6029b29b246b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 06 14:58:38.356795 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: warnings.warn(msg) May 06 14:58:38.356795 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: May 06 14:58:38.370739 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87094215-5cd1-4ac6-bc43-6029b29b246b admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:38.592044 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-87094215-5cd1-4ac6-bc43-6029b29b246b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a400d4befefc422c881ab8d3900a3d7e', 'request_id': 'req-87094215-5cd1-4ac6-bc43-6029b29b246b'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:38.604953 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-87094215-5cd1-4ac6-bc43-6029b29b246b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a400d4befefc422c881ab8d3900a3d7e', 'request_id': 'req-87094215-5cd1-4ac6-bc43-6029b29b246b'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:38.621930 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-87094215-5cd1-4ac6-bc43-6029b29b246b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a400d4befefc422c881ab8d3900a3d7e', 'request_id': 'req-87094215-5cd1-4ac6-bc43-6029b29b246b'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:38.660005 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 858/1696] 10.222.0.23 () {58 vars in 1018 bytes} [Fri May 6 14:58:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 807 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 14:58:38.660930 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb2c508-cd78-4ff5-8915-01922fe6af8e None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:38.678623 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-87094215-5cd1-4ac6-bc43-6029b29b246b admin admin] The token cache is being invalidate because project a400d4befefc422c881ab8d3900a3d7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 14:58:38.678880 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-87094215-5cd1-4ac6-bc43-6029b29b246b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:38.679779 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb2c508-cd78-4ff5-8915-01922fe6af8e None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:38.680161 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-87094215-5cd1-4ac6-bc43-6029b29b246b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:38.689470 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 839/1697] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 14:58:38 2022] DELETE /identity/v3/projects/a400d4befefc422c881ab8d3900a3d7e => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:38.691027 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef457125-b25b-4448-b17c-b1746d8e6128 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:38.691413 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef457125-b25b-4448-b17c-b1746d8e6128 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:38.691580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef457125-b25b-4448-b17c-b1746d8e6128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:38.815351 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-ef457125-b25b-4448-b17c-b1746d8e6128 None None] MFA Rules not processed for user `afd9f5519134415b8fc8a8c08149c502`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:38.817000 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ef457125-b25b-4448-b17c-b1746d8e6128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:39.041623 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d017e-0a60-4467-b11f-66b62ce1b3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:39.044167 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d017e-0a60-4467-b11f-66b62ce1b3bc admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:39.046428 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d017e-0a60-4467-b11f-66b62ce1b3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:39.047070 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d017e-0a60-4467-b11f-66b62ce1b3bc admin admin] PATH_INFO: `/v3/projects/1f5179c8b74b424092d8e3f6ec37316f/users/5c2494c6657946b89e4c7327dded6693/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:39.117428 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719d017e-0a60-4467-b11f-66b62ce1b3bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f5179c8b74b424092d8e3f6ec37316f, user_id=5c2494c6657946b89e4c7327dded6693, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:39.134267 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 840/1698] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 444 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:39.136818 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3ffc5-97eb-4a29-a5de-0d81ffa729f5 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:39.137115 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3ffc5-97eb-4a29-a5de-0d81ffa729f5 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:39.137397 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3ffc5-97eb-4a29-a5de-0d81ffa729f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:39.137892 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719d017e-0a60-4467-b11f-66b62ce1b3bc admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:39.196974 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 859/1699] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:58:38 2022] PUT /identity/v3/projects/1f5179c8b74b424092d8e3f6ec37316f/users/5c2494c6657946b89e4c7327dded6693/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:39.201055 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d017e-0a60-4467-b11f-66b62ce1b3bc admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:39.208855 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-719d017e-0a60-4467-b11f-66b62ce1b3bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:39.235117 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-d5e3ffc5-97eb-4a29-a5de-0d81ffa729f5 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:39.236401 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d5e3ffc5-97eb-4a29-a5de-0d81ffa729f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:39.386583 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 841/1700] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:39.388232 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be9b503-418d-4419-a27a-34d68adf0b56 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:39.393114 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be9b503-418d-4419-a27a-34d68adf0b56 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:39.393732 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be9b503-418d-4419-a27a-34d68adf0b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:39.498327 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-0be9b503-418d-4419-a27a-34d68adf0b56 None None] MFA Rules not processed for user `5c2494c6657946b89e4c7327dded6693`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:39.499544 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-0be9b503-418d-4419-a27a-34d68adf0b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:39.560443 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d017e-0a60-4467-b11f-66b62ce1b3bc admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:39.575354 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac30dfef-9fbd-4510-bfb2-4384fa2016a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:39.576035 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30dfef-9fbd-4510-bfb2-4384fa2016a2 service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:39.576325 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30dfef-9fbd-4510-bfb2-4384fa2016a2 service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:39.576620 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30dfef-9fbd-4510-bfb2-4384fa2016a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:39.581773 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-ac30dfef-9fbd-4510-bfb2-4384fa2016a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:39.614548 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac30dfef-9fbd-4510-bfb2-4384fa2016a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:39.619419 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac30dfef-9fbd-4510-bfb2-4384fa2016a2 service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:39.656042 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 860/1701] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:58:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:58:39.657342 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac30dfef-9fbd-4510-bfb2-4384fa2016a2 service nova] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:39.658863 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-ac30dfef-9fbd-4510-bfb2-4384fa2016a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:39.681506 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac30dfef-9fbd-4510-bfb2-4384fa2016a2 service nova] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:39.716937 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c4ba74-1b54-4ab5-af80-3584bf5de6de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:39.717752 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4ba74-1b54-4ab5-af80-3584bf5de6de service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:39.718041 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4ba74-1b54-4ab5-af80-3584bf5de6de service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:39.718325 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4ba74-1b54-4ab5-af80-3584bf5de6de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:39.782476 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 842/1702] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:39 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:39.783931 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2a623-34bb-453f-8c41-d947255b5e63 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:39.784226 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2a623-34bb-453f-8c41-d947255b5e63 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:39.784511 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2a623-34bb-453f-8c41-d947255b5e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:39.887111 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-c3f2a623-34bb-453f-8c41-d947255b5e63 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:39.888364 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-c3f2a623-34bb-453f-8c41-d947255b5e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:39.893438 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c4ba74-1b54-4ab5-af80-3584bf5de6de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:39.895325 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c4ba74-1b54-4ab5-af80-3584bf5de6de service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:39.925761 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 861/1703] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:58:39 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:58:39.926847 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c4ba74-1b54-4ab5-af80-3584bf5de6de service nova] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:39.933413 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 843/1704] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:39.937414 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f2a623-34bb-453f-8c41-d947255b5e63 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:39.938452 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-62c4ba74-1b54-4ab5-af80-3584bf5de6de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:40.001197 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f2a623-34bb-453f-8c41-d947255b5e63 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:40.006597 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c4ba74-1b54-4ab5-af80-3584bf5de6de service nova] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:40.025058 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4750da-e7d6-4f43-9913-5c5cfcfa04bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:40.026645 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4750da-e7d6-4f43-9913-5c5cfcfa04bd admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:40.027009 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9828035e-43a6-42e6-944a-fa5a9c7f47c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:40.027695 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828035e-43a6-42e6-944a-fa5a9c7f47c8 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:40.028031 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828035e-43a6-42e6-944a-fa5a9c7f47c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:40.028311 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828035e-43a6-42e6-944a-fa5a9c7f47c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:40.028811 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4750da-e7d6-4f43-9913-5c5cfcfa04bd admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:40.046749 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4750da-e7d6-4f43-9913-5c5cfcfa04bd admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:40.047027 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9828035e-43a6-42e6-944a-fa5a9c7f47c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:40.049181 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9828035e-43a6-42e6-944a-fa5a9c7f47c8 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:40.055230 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4750da-e7d6-4f43-9913-5c5cfcfa04bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:58:40.055902 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4750da-e7d6-4f43-9913-5c5cfcfa04bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:40.063856 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4750da-e7d6-4f43-9913-5c5cfcfa04bd admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:40.131630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 844/1705] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 14:58:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:58:40.137043 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4750da-e7d6-4f43-9913-5c5cfcfa04bd admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:40.180643 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4750da-e7d6-4f43-9913-5c5cfcfa04bd admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:40.184198 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 862/1706] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:58:39 2022] POST /identity/v3/projects => generated 367 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:58:40.190244 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9828035e-43a6-42e6-944a-fa5a9c7f47c8 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:40.377144 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793b9f12-c15f-4618-b3ac-c5bafc6e2bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:40.382281 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b9f12-c15f-4618-b3ac-c5bafc6e2bc0 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:40.382816 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b9f12-c15f-4618-b3ac-c5bafc6e2bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:40.383443 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b9f12-c15f-4618-b3ac-c5bafc6e2bc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:40.390780 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793b9f12-c15f-4618-b3ac-c5bafc6e2bc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:40.394928 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793b9f12-c15f-4618-b3ac-c5bafc6e2bc0 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:40.404484 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9828035e-43a6-42e6-944a-fa5a9c7f47c8 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:40.427995 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e05069-0a62-4883-ab1d-c6cb7b9f6558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:40.428833 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e05069-0a62-4883-ab1d-c6cb7b9f6558 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:40.432095 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e05069-0a62-4883-ab1d-c6cb7b9f6558 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:40.432399 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e05069-0a62-4883-ab1d-c6cb7b9f6558 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:40.433340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e05069-0a62-4883-ab1d-c6cb7b9f6558 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:40.438852 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e05069-0a62-4883-ab1d-c6cb7b9f6558 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:40.475092 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 845/1707] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:58:40 2022] POST /identity/v3/projects => generated 371 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 14:58:40.480589 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793b9f12-c15f-4618-b3ac-c5bafc6e2bc0 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:40.495564 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 863/1708] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:58:40 2022] POST /identity/v3/users => generated 290 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:58:40.500147 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e05069-0a62-4883-ab1d-c6cb7b9f6558 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:40.529776 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e05069-0a62-4883-ab1d-c6cb7b9f6558 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:40.587431 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793b9f12-c15f-4618-b3ac-c5bafc6e2bc0 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:40.600246 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c1d950-a9e9-4ac0-83e5-3b456067fdb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:40.601025 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1d950-a9e9-4ac0-83e5-3b456067fdb5 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:40.601263 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1d950-a9e9-4ac0-83e5-3b456067fdb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:40.601542 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1d950-a9e9-4ac0-83e5-3b456067fdb5 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:40.602280 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c1d950-a9e9-4ac0-83e5-3b456067fdb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:40.604438 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c1d950-a9e9-4ac0-83e5-3b456067fdb5 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:40.671086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a0b69-62a2-485e-b5e2-d6a9bdf88d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:40.672934 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a0b69-62a2-485e-b5e2-d6a9bdf88d7a admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:40.673320 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a0b69-62a2-485e-b5e2-d6a9bdf88d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:40.673611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a0b69-62a2-485e-b5e2-d6a9bdf88d7a admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:40.674398 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20a0b69-62a2-485e-b5e2-d6a9bdf88d7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:40.681192 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 846/1709] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:58:40 2022] POST /identity/v3/users => generated 286 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:58:40.684592 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20a0b69-62a2-485e-b5e2-d6a9bdf88d7a admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:40.689712 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c1d950-a9e9-4ac0-83e5-3b456067fdb5 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:40.719573 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 864/1710] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:40 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:58:40.726764 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a0b69-62a2-485e-b5e2-d6a9bdf88d7a admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:40.732741 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c1d950-a9e9-4ac0-83e5-3b456067fdb5 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:40.746896 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a0b69-62a2-485e-b5e2-d6a9bdf88d7a admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:40.755879 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6bd313-e13c-48c0-93ed-2015d88b17cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:40.756791 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6bd313-e13c-48c0-93ed-2015d88b17cd admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:40.758230 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6bd313-e13c-48c0-93ed-2015d88b17cd admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:40.758471 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6bd313-e13c-48c0-93ed-2015d88b17cd admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:40.762468 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c2b34-107c-4070-ae02-c8642cad3450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:40.764415 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c2b34-107c-4070-ae02-c8642cad3450 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:40.766232 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c2b34-107c-4070-ae02-c8642cad3450 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:40.766594 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c2b34-107c-4070-ae02-c8642cad3450 admin admin] PATH_INFO: `/v3/projects/d3f04cd2eb134745ae384a0e64cd03e0/users/7178c98d418f4b19a209dcdbf1ffa921/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:40.768145 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6bd313-e13c-48c0-93ed-2015d88b17cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:40.770023 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6bd313-e13c-48c0-93ed-2015d88b17cd admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:40.800321 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 847/1711] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:40 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:58:40.804455 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6bd313-e13c-48c0-93ed-2015d88b17cd admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:40.808968 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959c2b34-107c-4070-ae02-c8642cad3450 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3f04cd2eb134745ae384a0e64cd03e0, user_id=7178c98d418f4b19a209dcdbf1ffa921, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:40.814418 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959c2b34-107c-4070-ae02-c8642cad3450 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:40.822953 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6bd313-e13c-48c0-93ed-2015d88b17cd admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:40.834311 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb501b3-7cba-43bb-a245-5ae97b0973d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:40.835002 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb501b3-7cba-43bb-a245-5ae97b0973d9 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:40.835288 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb501b3-7cba-43bb-a245-5ae97b0973d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:40.835567 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb501b3-7cba-43bb-a245-5ae97b0973d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:40.836296 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb501b3-7cba-43bb-a245-5ae97b0973d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:40.856665 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 865/1712] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:58:40 2022] PUT /identity/v3/projects/d3f04cd2eb134745ae384a0e64cd03e0/users/7178c98d418f4b19a209dcdbf1ffa921/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:40.860337 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb501b3-7cba-43bb-a245-5ae97b0973d9 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:40.861311 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c2b34-107c-4070-ae02-c8642cad3450 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:40.899813 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 848/1713] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:40 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:58:40.904898 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb501b3-7cba-43bb-a245-5ae97b0973d9 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:40.923517 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb501b3-7cba-43bb-a245-5ae97b0973d9 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:41.089917 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3ec228-fb4b-41ee-8328-3382ae0c1641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:41.090843 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ec228-fb4b-41ee-8328-3382ae0c1641 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:41.091266 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ec228-fb4b-41ee-8328-3382ae0c1641 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:41.091765 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ec228-fb4b-41ee-8328-3382ae0c1641 admin admin] PATH_INFO: `/v3/projects/4d1150f1e7b842c9a1f29aa9777fada8/users/28d224ed22b545a1bebc50a8d465075b/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:41.096218 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c2b34-107c-4070-ae02-c8642cad3450 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:41.112570 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ebf41-6769-47d0-9a44-3e307ba6b46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:41.115888 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ebf41-6769-47d0-9a44-3e307ba6b46d admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:41.118781 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ebf41-6769-47d0-9a44-3e307ba6b46d admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:41.118990 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ebf41-6769-47d0-9a44-3e307ba6b46d admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:41.121490 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318ebf41-6769-47d0-9a44-3e307ba6b46d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:41.126307 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3ec228-fb4b-41ee-8328-3382ae0c1641 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d1150f1e7b842c9a1f29aa9777fada8, user_id=28d224ed22b545a1bebc50a8d465075b, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:41.129614 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3ec228-fb4b-41ee-8328-3382ae0c1641 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:41.140126 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318ebf41-6769-47d0-9a44-3e307ba6b46d admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:41.166658 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 849/1714] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:58:40 2022] PUT /identity/v3/projects/4d1150f1e7b842c9a1f29aa9777fada8/users/28d224ed22b545a1bebc50a8d465075b/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 14:58:41.177058 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b67832-634e-48b1-8381-2bae091493a3 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:41.177731 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b67832-634e-48b1-8381-2bae091493a3 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:41.178282 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b67832-634e-48b1-8381-2bae091493a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:41.203979 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 866/1715] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:40 2022] GET /identity/v3/roles => generated 1420 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:58:41.206509 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ebf41-6769-47d0-9a44-3e307ba6b46d admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:41.240997 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ebf41-6769-47d0-9a44-3e307ba6b46d admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:41.259457 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-86b67832-634e-48b1-8381-2bae091493a3 None None] MFA Rules not processed for user `28d224ed22b545a1bebc50a8d465075b`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:41.266149 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-86b67832-634e-48b1-8381-2bae091493a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:41.451280 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce335d2-dbc0-4d2e-9a01-6e1d7ab2d805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:41.452940 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce335d2-dbc0-4d2e-9a01-6e1d7ab2d805 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:41.454815 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce335d2-dbc0-4d2e-9a01-6e1d7ab2d805 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:41.456989 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce335d2-dbc0-4d2e-9a01-6e1d7ab2d805 admin admin] PATH_INFO: `/v3/projects/d3f04cd2eb134745ae384a0e64cd03e0/users/7178c98d418f4b19a209dcdbf1ffa921/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:41.469668 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce335d2-dbc0-4d2e-9a01-6e1d7ab2d805 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3f04cd2eb134745ae384a0e64cd03e0, user_id=7178c98d418f4b19a209dcdbf1ffa921, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:41.473523 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce335d2-dbc0-4d2e-9a01-6e1d7ab2d805 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:41.506007 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 867/1716] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:58:41 2022] PUT /identity/v3/projects/d3f04cd2eb134745ae384a0e64cd03e0/users/7178c98d418f4b19a209dcdbf1ffa921/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:41.525999 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e73dd-179b-48d2-8982-ae3cff613d18 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:41.526690 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e73dd-179b-48d2-8982-ae3cff613d18 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:41.526993 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e73dd-179b-48d2-8982-ae3cff613d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:41.533384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 850/1717] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 14:58:41.538715 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b67832-634e-48b1-8381-2bae091493a3 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:41.566725 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b67832-634e-48b1-8381-2bae091493a3 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:41.603004 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-0e9e73dd-179b-48d2-8982-ae3cff613d18 None None] MFA Rules not processed for user `7178c98d418f4b19a209dcdbf1ffa921`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:41.609710 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-0e9e73dd-179b-48d2-8982-ae3cff613d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:41.700010 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf760f1-8575-464e-9264-07925c4bf648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:41.701217 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf760f1-8575-464e-9264-07925c4bf648 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:41.701403 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf760f1-8575-464e-9264-07925c4bf648 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:41.701875 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf760f1-8575-464e-9264-07925c4bf648 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:41.704328 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf760f1-8575-464e-9264-07925c4bf648 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:41.709901 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf760f1-8575-464e-9264-07925c4bf648 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:41.786481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 851/1718] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:58:41 2022] POST /identity/v3/projects => generated 371 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:58:41.792709 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3891e34-449a-48d2-9735-bcdd9eadd20d None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:41.793425 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3891e34-449a-48d2-9735-bcdd9eadd20d None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:41.794451 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3891e34-449a-48d2-9735-bcdd9eadd20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:41.887025 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-d3891e34-449a-48d2-9735-bcdd9eadd20d None None] MFA Rules not processed for user `02b07e776b2c4bfba6eabb15999aec35`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:41.893079 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 868/1719] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:41 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:41.897847 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9e73dd-179b-48d2-8982-ae3cff613d18 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:41.902182 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d3891e34-449a-48d2-9735-bcdd9eadd20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:41.959972 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9e73dd-179b-48d2-8982-ae3cff613d18 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:42.205710 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 852/1720] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 418 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:42.211423 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3891e34-449a-48d2-9735-bcdd9eadd20d None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:42.218922 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d3891e34-449a-48d2-9735-bcdd9eadd20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:42.230509 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9759772-48ce-4792-9904-a99fd0588d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:42.231335 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9759772-48ce-4792-9904-a99fd0588d5a admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:42.231670 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9759772-48ce-4792-9904-a99fd0588d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:42.232015 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9759772-48ce-4792-9904-a99fd0588d5a admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:42.238589 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9759772-48ce-4792-9904-a99fd0588d5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:42.249375 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9759772-48ce-4792-9904-a99fd0588d5a admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:42.313126 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 869/1721] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:58:41 2022] POST /identity/v3/users => generated 292 bytes in 419 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:58:42.313874 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9759772-48ce-4792-9904-a99fd0588d5a admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:42.486938 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3891e34-449a-48d2-9735-bcdd9eadd20d None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:42.492114 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1cfcf4-df08-4fa8-9173-58a755efd6d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:42.492978 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1cfcf4-df08-4fa8-9173-58a755efd6d1 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:42.493416 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1cfcf4-df08-4fa8-9173-58a755efd6d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:42.493744 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1cfcf4-df08-4fa8-9173-58a755efd6d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:42.574501 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9759772-48ce-4792-9904-a99fd0588d5a admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:42.591493 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd17dc4-dac4-491a-b0b9-a4faff8f69b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:42.592183 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd17dc4-dac4-491a-b0b9-a4faff8f69b7 service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:42.592464 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd17dc4-dac4-491a-b0b9-a4faff8f69b7 service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:42.592763 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd17dc4-dac4-491a-b0b9-a4faff8f69b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:42.615532 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd17dc4-dac4-491a-b0b9-a4faff8f69b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:42.617185 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd17dc4-dac4-491a-b0b9-a4faff8f69b7 service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:42.642647 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 870/1722] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:58:42 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:58:42.643726 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd17dc4-dac4-491a-b0b9-a4faff8f69b7 service nova] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:42.656267 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1cfcf4-df08-4fa8-9173-58a755efd6d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:42.658522 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1cfcf4-df08-4fa8-9173-58a755efd6d1 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:42.671130 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd17dc4-dac4-491a-b0b9-a4faff8f69b7 service nova] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:42.684237 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ad172-159b-4db5-8c82-5a60c0ea2c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:42.685391 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ad172-159b-4db5-8c82-5a60c0ea2c3d admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:42.685820 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ad172-159b-4db5-8c82-5a60c0ea2c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:42.686420 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ad172-159b-4db5-8c82-5a60c0ea2c3d admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:42.687942 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7ad172-159b-4db5-8c82-5a60c0ea2c3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:42.690214 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7ad172-159b-4db5-8c82-5a60c0ea2c3d admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:42.690788 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 853/1723] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:58:42 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:58:42.741828 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1cfcf4-df08-4fa8-9173-58a755efd6d1 service cinder] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:42.747946 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 871/1724] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:42 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:58:42.757216 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1cfcf4-df08-4fa8-9173-58a755efd6d1 service cinder] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:42.766149 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d6aeb4-d255-40d6-ad42-ec9e96acbfe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:42.767057 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d6aeb4-d255-40d6-ad42-ec9e96acbfe6 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:42.767335 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d6aeb4-d255-40d6-ad42-ec9e96acbfe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:42.767609 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d6aeb4-d255-40d6-ad42-ec9e96acbfe6 admin admin] PATH_INFO: `/v3/projects/a5fa23f308fd45f88add04f184ccd2b8/users/0ab2746221ae44e8b9e359aa2f587ffb/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:42.809503 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d6aeb4-d255-40d6-ad42-ec9e96acbfe6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5fa23f308fd45f88add04f184ccd2b8, user_id=0ab2746221ae44e8b9e359aa2f587ffb, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:42.812192 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d6aeb4-d255-40d6-ad42-ec9e96acbfe6 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:42.860575 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ad172-159b-4db5-8c82-5a60c0ea2c3d admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:42.861052 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 854/1725] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:58:42 2022] PUT /identity/v3/projects/a5fa23f308fd45f88add04f184ccd2b8/users/0ab2746221ae44e8b9e359aa2f587ffb/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:42.878123 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ad172-159b-4db5-8c82-5a60c0ea2c3d admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:42.987163 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d6aeb4-d255-40d6-ad42-ec9e96acbfe6 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:43.065402 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc328e9-cda2-41fb-b379-7accf7418b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:43.066266 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc328e9-cda2-41fb-b379-7accf7418b6a admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:43.066544 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc328e9-cda2-41fb-b379-7accf7418b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:43.066859 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc328e9-cda2-41fb-b379-7accf7418b6a admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:43.067540 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc328e9-cda2-41fb-b379-7accf7418b6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:43.069500 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc328e9-cda2-41fb-b379-7accf7418b6a admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:43.123824 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 872/1726] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:42 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:58:43.129133 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc328e9-cda2-41fb-b379-7accf7418b6a admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:43.171067 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc328e9-cda2-41fb-b379-7accf7418b6a admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:43.180125 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d6aeb4-d255-40d6-ad42-ec9e96acbfe6 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:43.196868 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320c5da3-d8db-480b-a524-61e385b40825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:43.200841 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e679b807-3247-4d73-9977-1611c71fc1be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:43.201728 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679b807-3247-4d73-9977-1611c71fc1be service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:43.203156 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c5da3-d8db-480b-a524-61e385b40825 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:43.204347 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679b807-3247-4d73-9977-1611c71fc1be service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:43.204809 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c5da3-d8db-480b-a524-61e385b40825 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:43.205239 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679b807-3247-4d73-9977-1611c71fc1be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:43.207522 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c5da3-d8db-480b-a524-61e385b40825 admin admin] PATH_INFO: `/v3/projects/a5fa23f308fd45f88add04f184ccd2b8/users/0ab2746221ae44e8b9e359aa2f587ffb/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:43.219403 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320c5da3-d8db-480b-a524-61e385b40825 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5fa23f308fd45f88add04f184ccd2b8, user_id=0ab2746221ae44e8b9e359aa2f587ffb, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:43.227359 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320c5da3-d8db-480b-a524-61e385b40825 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:43.302218 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 873/1727] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:58:43 2022] PUT /identity/v3/projects/a5fa23f308fd45f88add04f184ccd2b8/users/0ab2746221ae44e8b9e359aa2f587ffb/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:43.305076 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f35baf-ede3-4c5f-9dd0-21c2087037e6 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:43.305481 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f35baf-ede3-4c5f-9dd0-21c2087037e6 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:43.305867 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f35baf-ede3-4c5f-9dd0-21c2087037e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:43.428486 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-a7f35baf-ede3-4c5f-9dd0-21c2087037e6 None None] MFA Rules not processed for user `81c9e87c348c4a2c82e8327a2cadfc5d`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:43.432268 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-a7f35baf-ede3-4c5f-9dd0-21c2087037e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:43.536159 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e679b807-3247-4d73-9977-1611c71fc1be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:43.542588 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e679b807-3247-4d73-9977-1611c71fc1be service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:43.578844 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 855/1728] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:58:42 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:58:43.581682 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9fc30-751a-4671-8f16-64651274ee90 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:43.586590 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9fc30-751a-4671-8f16-64651274ee90 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:43.587104 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9fc30-751a-4671-8f16-64651274ee90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:43.665076 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 874/1729] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:43.684447 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efef049-a4a0-45b5-9618-05b9675dacee None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:43.684767 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efef049-a4a0-45b5-9618-05b9675dacee None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:43.685047 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efef049-a4a0-45b5-9618-05b9675dacee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:43.692182 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-1cd9fc30-751a-4671-8f16-64651274ee90 None None] MFA Rules not processed for user `0ab2746221ae44e8b9e359aa2f587ffb`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:43.696340 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-1cd9fc30-751a-4671-8f16-64651274ee90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:43.723803 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-2efef049-a4a0-45b5-9618-05b9675dacee None None] MFA Rules not processed for user `81c9e87c348c4a2c82e8327a2cadfc5d`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:43.727966 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-2efef049-a4a0-45b5-9618-05b9675dacee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:43.774339 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 875/1730] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 14:58:43.789781 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efef049-a4a0-45b5-9618-05b9675dacee None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:44.083616 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efef049-a4a0-45b5-9618-05b9675dacee None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:44.104370 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fee39b-4912-4b08-8156-546da6af9222 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:44.105565 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fee39b-4912-4b08-8156-546da6af9222 service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:44.108953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fee39b-4912-4b08-8156-546da6af9222 service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:44.109377 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fee39b-4912-4b08-8156-546da6af9222 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:44.133820 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fee39b-4912-4b08-8156-546da6af9222 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:44.140089 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fee39b-4912-4b08-8156-546da6af9222 service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:44.164700 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 856/1731] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:43 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 581 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:44.179822 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 876/1732] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:58:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:58:44.334292 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9fc30-751a-4671-8f16-64651274ee90 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:44.487013 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9fc30-751a-4671-8f16-64651274ee90 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:44.508386 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81eded9-57df-437b-b4d2-0bab764fd482 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:44.509157 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81eded9-57df-437b-b4d2-0bab764fd482 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:44.509566 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81eded9-57df-437b-b4d2-0bab764fd482 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:44.513705 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81eded9-57df-437b-b4d2-0bab764fd482 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:44.545494 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81eded9-57df-437b-b4d2-0bab764fd482 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:44.553667 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81eded9-57df-437b-b4d2-0bab764fd482 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:44.612981 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 857/1733] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:58:44 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:58:47.421422 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802c280-7b62-4ea0-b825-91800218f76d None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:47.422632 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802c280-7b62-4ea0-b825-91800218f76d None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:47.423202 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802c280-7b62-4ea0-b825-91800218f76d None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:47.424833 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 877/1734] 10.222.0.23 () {54 vars in 912 bytes} [Fri May 6 14:58:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:58:47.430801 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02331130-0a57-4713-9982-36d9f809899f None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:47.431913 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02331130-0a57-4713-9982-36d9f809899f None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:47.432080 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02331130-0a57-4713-9982-36d9f809899f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:47.476670 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-02331130-0a57-4713-9982-36d9f809899f None None] MFA Rules not processed for user `4dcd1542d8f643c4996142a88b9b6abb`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:47.482045 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-02331130-0a57-4713-9982-36d9f809899f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:47.636107 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 858/1735] 10.222.0.23 () {58 vars in 1009 bytes} [Fri May 6 14:58:47 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:47.659854 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e802c280-7b62-4ea0-b825-91800218f76d None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:47.665400 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e802c280-7b62-4ea0-b825-91800218f76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:47.704997 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e802c280-7b62-4ea0-b825-91800218f76d None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:47.710901 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed80c9f-b78d-4042-8c3a-53dc4557fa02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:47.712091 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed80c9f-b78d-4042-8c3a-53dc4557fa02 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:47.713729 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed80c9f-b78d-4042-8c3a-53dc4557fa02 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:47.714370 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed80c9f-b78d-4042-8c3a-53dc4557fa02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:47.733605 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02331130-0a57-4713-9982-36d9f809899f None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:47.740116 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed80c9f-b78d-4042-8c3a-53dc4557fa02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:47.746191 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed80c9f-b78d-4042-8c3a-53dc4557fa02 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:47.783871 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 878/1736] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:58:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:58:47.897723 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02331130-0a57-4713-9982-36d9f809899f None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:47.907846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f018a60-e350-40fb-9dfa-26310ec18f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:47.908897 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f018a60-e350-40fb-9dfa-26310ec18f4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:47.909044 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f018a60-e350-40fb-9dfa-26310ec18f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:47.909474 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f018a60-e350-40fb-9dfa-26310ec18f4b admin admin] PATH_INFO: `/v3/users/28d224ed22b545a1bebc50a8d465075b` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:47.911386 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f018a60-e350-40fb-9dfa-26310ec18f4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=28d224ed22b545a1bebc50a8d465075b)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:47.913514 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f018a60-e350-40fb-9dfa-26310ec18f4b admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:48.042746 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-0f018a60-e350-40fb-9dfa-26310ec18f4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28d224ed22b545a1bebc50a8d465075b'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:48.045080 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-0f018a60-e350-40fb-9dfa-26310ec18f4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28d224ed22b545a1bebc50a8d465075b'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:48.058690 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-0f018a60-e350-40fb-9dfa-26310ec18f4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28d224ed22b545a1bebc50a8d465075b'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:48.081077 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-0f018a60-e350-40fb-9dfa-26310ec18f4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28d224ed22b545a1bebc50a8d465075b'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:48.082752 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-0f018a60-e350-40fb-9dfa-26310ec18f4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28d224ed22b545a1bebc50a8d465075b'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:48.129503 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 859/1737] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 14:58:47 2022] DELETE /identity/v3/users/28d224ed22b545a1bebc50a8d465075b => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 14:58:48.136334 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c9e10-c36a-472e-90ee-786a0eabacca None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:48.136685 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c9e10-c36a-472e-90ee-786a0eabacca None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:48.137015 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c9e10-c36a-472e-90ee-786a0eabacca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:48.166426 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f018a60-e350-40fb-9dfa-26310ec18f4b admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:48.169518 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-0f018a60-e350-40fb-9dfa-26310ec18f4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:48.192693 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-726c9e10-c36a-472e-90ee-786a0eabacca None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:48.195394 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-726c9e10-c36a-472e-90ee-786a0eabacca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:48.365380 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f018a60-e350-40fb-9dfa-26310ec18f4b admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:48.374911 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b0a7ba-514a-4d2d-a2a5-6cd283c2f2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:48.375802 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0a7ba-514a-4d2d-a2a5-6cd283c2f2c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:48.376778 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0a7ba-514a-4d2d-a2a5-6cd283c2f2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:48.376778 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0a7ba-514a-4d2d-a2a5-6cd283c2f2c0 admin admin] PATH_INFO: `/v3/users/025d805b26d84e3994a9d17660372016` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:48.382209 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b0a7ba-514a-4d2d-a2a5-6cd283c2f2c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=025d805b26d84e3994a9d17660372016)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:48.384381 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b0a7ba-514a-4d2d-a2a5-6cd283c2f2c0 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:48.440594 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 879/1738] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:48.463605 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c9e10-c36a-472e-90ee-786a0eabacca None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:48.467698 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-726c9e10-c36a-472e-90ee-786a0eabacca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:48.494975 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-72b0a7ba-514a-4d2d-a2a5-6cd283c2f2c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '025d805b26d84e3994a9d17660372016'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:48.496020 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-72b0a7ba-514a-4d2d-a2a5-6cd283c2f2c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '025d805b26d84e3994a9d17660372016'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:48.507619 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-72b0a7ba-514a-4d2d-a2a5-6cd283c2f2c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '025d805b26d84e3994a9d17660372016'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:48.523380 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-72b0a7ba-514a-4d2d-a2a5-6cd283c2f2c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '025d805b26d84e3994a9d17660372016'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:48.527052 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-72b0a7ba-514a-4d2d-a2a5-6cd283c2f2c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '025d805b26d84e3994a9d17660372016'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:48.596026 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 860/1739] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 14:58:48 2022] DELETE /identity/v3/users/025d805b26d84e3994a9d17660372016 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:48.602227 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2618cb1-4064-4d95-8b72-f35af6c1f5e2 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:48.602554 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2618cb1-4064-4d95-8b72-f35af6c1f5e2 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:48.602831 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2618cb1-4064-4d95-8b72-f35af6c1f5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:48.663344 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c9e10-c36a-472e-90ee-786a0eabacca None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:48.692204 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-b2618cb1-4064-4d95-8b72-f35af6c1f5e2 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:48.709702 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b2618cb1-4064-4d95-8b72-f35af6c1f5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:48.825083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c123639-9be9-4f14-ac2d-37c323699593 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:48.826080 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c123639-9be9-4f14-ac2d-37c323699593 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:48.826532 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c123639-9be9-4f14-ac2d-37c323699593 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:48.826996 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c123639-9be9-4f14-ac2d-37c323699593 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:48.848721 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-4c123639-9be9-4f14-ac2d-37c323699593 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:48.911958 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 861/1740] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:48.925030 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5828c63-a3fb-4cb2-a184-6541a80b3ccb None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:48.925479 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5828c63-a3fb-4cb2-a184-6541a80b3ccb None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:48.926107 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5828c63-a3fb-4cb2-a184-6541a80b3ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:48.981360 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-c5828c63-a3fb-4cb2-a184-6541a80b3ccb None None] MFA Rules not processed for user `afd9f5519134415b8fc8a8c08149c502`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:48.982835 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-c5828c63-a3fb-4cb2-a184-6541a80b3ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:49.110801 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c123639-9be9-4f14-ac2d-37c323699593 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:49.115556 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c123639-9be9-4f14-ac2d-37c323699593 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:49.145409 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 880/1741] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:58:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 684 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:58:49.146956 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c123639-9be9-4f14-ac2d-37c323699593 service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:49.154490 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-4c123639-9be9-4f14-ac2d-37c323699593 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:49.184456 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c123639-9be9-4f14-ac2d-37c323699593 service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:49.187055 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 862/1742] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:49.192412 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d64b8-0e10-4980-9e4c-4f702be496ff None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:49.192412 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d64b8-0e10-4980-9e4c-4f702be496ff None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:49.192412 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d64b8-0e10-4980-9e4c-4f702be496ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:49.193133 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a908790-d6fd-4cc1-bcfc-e7045b3ce0a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:49.193997 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a908790-d6fd-4cc1-bcfc-e7045b3ce0a2 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:49.197459 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a908790-d6fd-4cc1-bcfc-e7045b3ce0a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:49.201649 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a908790-d6fd-4cc1-bcfc-e7045b3ce0a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:49.226369 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a908790-d6fd-4cc1-bcfc-e7045b3ce0a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:49.228435 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a908790-d6fd-4cc1-bcfc-e7045b3ce0a2 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:49.255695 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 881/1743] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:58:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:58:49.258599 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b37b05-c344-48ff-881b-fef9d7a95914 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:49.258895 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b37b05-c344-48ff-881b-fef9d7a95914 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:49.259224 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b37b05-c344-48ff-881b-fef9d7a95914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:49.286386 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-407d64b8-0e10-4980-9e4c-4f702be496ff None None] MFA Rules not processed for user `81c9e87c348c4a2c82e8327a2cadfc5d`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:49.289420 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-407d64b8-0e10-4980-9e4c-4f702be496ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:49.315893 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-a6b37b05-c344-48ff-881b-fef9d7a95914 None None] MFA Rules not processed for user `02b07e776b2c4bfba6eabb15999aec35`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:49.347501 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-a6b37b05-c344-48ff-881b-fef9d7a95914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:49.439904 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 863/1744] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:49.560164 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 882/1745] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:50.706952 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d64b8-0e10-4980-9e4c-4f702be496ff None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:50.708398 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-407d64b8-0e10-4980-9e4c-4f702be496ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:50.745958 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d64b8-0e10-4980-9e4c-4f702be496ff None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:50.769373 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b3360-db14-4569-8899-42670a545684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:50.771180 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b3360-db14-4569-8899-42670a545684 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:50.771463 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b3360-db14-4569-8899-42670a545684 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:50.771742 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b3360-db14-4569-8899-42670a545684 admin admin] PATH_INFO: `/v3/projects/1b8d2d8b71924fe8a56edf41750ca99b` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:50.775559 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b3360-db14-4569-8899-42670a545684 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b8d2d8b71924fe8a56edf41750ca99b)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:50.781758 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b3360-db14-4569-8899-42670a545684 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:50.909578 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-cb6b3360-db14-4569-8899-42670a545684 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b8d2d8b71924fe8a56edf41750ca99b', 'request_id': 'req-cb6b3360-db14-4569-8899-42670a545684'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:50.926815 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-cb6b3360-db14-4569-8899-42670a545684 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b8d2d8b71924fe8a56edf41750ca99b', 'request_id': 'req-cb6b3360-db14-4569-8899-42670a545684'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:50.940363 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-cb6b3360-db14-4569-8899-42670a545684 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b8d2d8b71924fe8a56edf41750ca99b', 'request_id': 'req-cb6b3360-db14-4569-8899-42670a545684'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:50.976546 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-cb6b3360-db14-4569-8899-42670a545684 admin admin] The token cache is being invalidate because project 1b8d2d8b71924fe8a56edf41750ca99b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 14:58:50.981513 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-cb6b3360-db14-4569-8899-42670a545684 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:50.986104 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-cb6b3360-db14-4569-8899-42670a545684 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:50.994382 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 864/1746] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 14:58:50 2022] DELETE /identity/v3/projects/1b8d2d8b71924fe8a56edf41750ca99b => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:50.995630 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b37b05-c344-48ff-881b-fef9d7a95914 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:51.011237 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b37b05-c344-48ff-881b-fef9d7a95914 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:51.049163 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b3360-db14-4569-8899-42670a545684 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:51.051257 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-cb6b3360-db14-4569-8899-42670a545684 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:51.218231 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b3360-db14-4569-8899-42670a545684 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:51.223864 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5469aab-6165-4ca4-8e63-396a6a1fff7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:51.225582 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5469aab-6165-4ca4-8e63-396a6a1fff7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:51.226029 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5469aab-6165-4ca4-8e63-396a6a1fff7d admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:51.226471 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5469aab-6165-4ca4-8e63-396a6a1fff7d admin admin] PATH_INFO: `/v3/projects/4d1150f1e7b842c9a1f29aa9777fada8` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:51.228777 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5469aab-6165-4ca4-8e63-396a6a1fff7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d1150f1e7b842c9a1f29aa9777fada8)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:51.231230 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5469aab-6165-4ca4-8e63-396a6a1fff7d admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:51.232788 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed25c938-0451-4d25-8194-1825d0874965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:51.234002 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25c938-0451-4d25-8194-1825d0874965 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:51.236936 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25c938-0451-4d25-8194-1825d0874965 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:51.237390 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25c938-0451-4d25-8194-1825d0874965 admin admin] PATH_INFO: `/v3/users/bf31486468904557ae04cdbf7de06b70` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:51.239467 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed25c938-0451-4d25-8194-1825d0874965 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf31486468904557ae04cdbf7de06b70)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:51.246401 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed25c938-0451-4d25-8194-1825d0874965 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:51.333537 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d5469aab-6165-4ca4-8e63-396a6a1fff7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d1150f1e7b842c9a1f29aa9777fada8', 'request_id': 'req-d5469aab-6165-4ca4-8e63-396a6a1fff7d'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:51.343165 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d5469aab-6165-4ca4-8e63-396a6a1fff7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d1150f1e7b842c9a1f29aa9777fada8', 'request_id': 'req-d5469aab-6165-4ca4-8e63-396a6a1fff7d'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:51.345980 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ed25c938-0451-4d25-8194-1825d0874965 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf31486468904557ae04cdbf7de06b70'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:51.358690 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d5469aab-6165-4ca4-8e63-396a6a1fff7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d1150f1e7b842c9a1f29aa9777fada8', 'request_id': 'req-d5469aab-6165-4ca4-8e63-396a6a1fff7d'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:51.363453 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ed25c938-0451-4d25-8194-1825d0874965 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf31486468904557ae04cdbf7de06b70'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:51.378514 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ed25c938-0451-4d25-8194-1825d0874965 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf31486468904557ae04cdbf7de06b70'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:51.379487 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ed25c938-0451-4d25-8194-1825d0874965 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf31486468904557ae04cdbf7de06b70'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:51.380354 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ed25c938-0451-4d25-8194-1825d0874965 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf31486468904557ae04cdbf7de06b70'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:51.382655 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d5469aab-6165-4ca4-8e63-396a6a1fff7d admin admin] The token cache is being invalidate because project 4d1150f1e7b842c9a1f29aa9777fada8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 14:58:51.382655 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d5469aab-6165-4ca4-8e63-396a6a1fff7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:51.383548 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d5469aab-6165-4ca4-8e63-396a6a1fff7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:51.386651 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 865/1747] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 14:58:51 2022] DELETE /identity/v3/projects/4d1150f1e7b842c9a1f29aa9777fada8 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 14:58:51.399014 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5469aab-6165-4ca4-8e63-396a6a1fff7d admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:51.401838 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d5469aab-6165-4ca4-8e63-396a6a1fff7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:51.423601 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 883/1748] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 14:58:50 2022] DELETE /identity/v3/users/bf31486468904557ae04cdbf7de06b70 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:51.532523 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5469aab-6165-4ca4-8e63-396a6a1fff7d admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:51.663383 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e101f7-f2e2-475e-ba5e-9b3b2d98e19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:51.664972 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e101f7-f2e2-475e-ba5e-9b3b2d98e19e admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:51.665363 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e101f7-f2e2-475e-ba5e-9b3b2d98e19e admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:51.666879 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e101f7-f2e2-475e-ba5e-9b3b2d98e19e admin admin] PATH_INFO: `/v3/users/0ab2746221ae44e8b9e359aa2f587ffb` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:51.670077 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e101f7-f2e2-475e-ba5e-9b3b2d98e19e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ab2746221ae44e8b9e359aa2f587ffb)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:51.673569 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e101f7-f2e2-475e-ba5e-9b3b2d98e19e admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:51.801771 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-54e101f7-f2e2-475e-ba5e-9b3b2d98e19e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ab2746221ae44e8b9e359aa2f587ffb'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:51.802942 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-54e101f7-f2e2-475e-ba5e-9b3b2d98e19e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ab2746221ae44e8b9e359aa2f587ffb'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:51.815446 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-54e101f7-f2e2-475e-ba5e-9b3b2d98e19e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ab2746221ae44e8b9e359aa2f587ffb'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:51.826882 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-54e101f7-f2e2-475e-ba5e-9b3b2d98e19e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ab2746221ae44e8b9e359aa2f587ffb'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:51.828458 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-54e101f7-f2e2-475e-ba5e-9b3b2d98e19e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ab2746221ae44e8b9e359aa2f587ffb'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:51.852242 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 866/1749] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 14:58:51 2022] DELETE /identity/v3/users/0ab2746221ae44e8b9e359aa2f587ffb => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:52.324807 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed25c938-0451-4d25-8194-1825d0874965 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:52.327325 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-ed25c938-0451-4d25-8194-1825d0874965 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:52.350158 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed25c938-0451-4d25-8194-1825d0874965 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:52.504594 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b783b5f-90c3-4482-a0c2-92ba495ad910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:52.505845 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b783b5f-90c3-4482-a0c2-92ba495ad910 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:52.506206 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b783b5f-90c3-4482-a0c2-92ba495ad910 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:52.506561 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b783b5f-90c3-4482-a0c2-92ba495ad910 admin admin] PATH_INFO: `/v3/projects/9975f78865eb4440a2587efc16fa7283` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:52.508774 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b783b5f-90c3-4482-a0c2-92ba495ad910 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9975f78865eb4440a2587efc16fa7283)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:52.511977 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b783b5f-90c3-4482-a0c2-92ba495ad910 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:52.558275 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e101f7-f2e2-475e-ba5e-9b3b2d98e19e admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:52.560243 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-54e101f7-f2e2-475e-ba5e-9b3b2d98e19e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:52.587090 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e101f7-f2e2-475e-ba5e-9b3b2d98e19e admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:52.601122 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77f9f48-2970-4756-8635-ef36afb877c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:52.602041 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77f9f48-2970-4756-8635-ef36afb877c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:52.602378 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77f9f48-2970-4756-8635-ef36afb877c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:52.602580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77f9f48-2970-4756-8635-ef36afb877c4 admin admin] PATH_INFO: `/v3/users/5c2494c6657946b89e4c7327dded6693` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:52.606490 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77f9f48-2970-4756-8635-ef36afb877c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c2494c6657946b89e4c7327dded6693)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:52.613148 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77f9f48-2970-4756-8635-ef36afb877c4 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:52.626522 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-7b783b5f-90c3-4482-a0c2-92ba495ad910 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9975f78865eb4440a2587efc16fa7283', 'request_id': 'req-7b783b5f-90c3-4482-a0c2-92ba495ad910'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:52.644287 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-7b783b5f-90c3-4482-a0c2-92ba495ad910 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9975f78865eb4440a2587efc16fa7283', 'request_id': 'req-7b783b5f-90c3-4482-a0c2-92ba495ad910'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:52.665039 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-7b783b5f-90c3-4482-a0c2-92ba495ad910 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9975f78865eb4440a2587efc16fa7283', 'request_id': 'req-7b783b5f-90c3-4482-a0c2-92ba495ad910'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:52.704307 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-7b783b5f-90c3-4482-a0c2-92ba495ad910 admin admin] The token cache is being invalidate because project 9975f78865eb4440a2587efc16fa7283 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 14:58:52.706223 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-7b783b5f-90c3-4482-a0c2-92ba495ad910 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:52.708737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-7b783b5f-90c3-4482-a0c2-92ba495ad910 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:52.718488 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 884/1750] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 14:58:52 2022] DELETE /identity/v3/projects/9975f78865eb4440a2587efc16fa7283 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:52.719755 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b783b5f-90c3-4482-a0c2-92ba495ad910 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:52.724457 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-7b783b5f-90c3-4482-a0c2-92ba495ad910 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:52.734676 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-c77f9f48-2970-4756-8635-ef36afb877c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c2494c6657946b89e4c7327dded6693'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:52.742751 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-c77f9f48-2970-4756-8635-ef36afb877c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c2494c6657946b89e4c7327dded6693'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:52.759417 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-c77f9f48-2970-4756-8635-ef36afb877c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c2494c6657946b89e4c7327dded6693'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:52.769380 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-c77f9f48-2970-4756-8635-ef36afb877c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c2494c6657946b89e4c7327dded6693'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:52.770792 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-c77f9f48-2970-4756-8635-ef36afb877c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c2494c6657946b89e4c7327dded6693'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:52.813336 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 867/1751] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 14:58:52 2022] DELETE /identity/v3/users/5c2494c6657946b89e4c7327dded6693 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 14:58:52.815339 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced8329-f535-4a37-9322-2248d3c3aaa6 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:52.815792 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced8329-f535-4a37-9322-2248d3c3aaa6 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:52.816200 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced8329-f535-4a37-9322-2248d3c3aaa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:52.838369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-7ced8329-f535-4a37-9322-2248d3c3aaa6 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:52.840162 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-7ced8329-f535-4a37-9322-2248d3c3aaa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:52.871539 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b783b5f-90c3-4482-a0c2-92ba495ad910 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:52.970637 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 868/1752] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:52.972498 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15ef0f-d3b1-47a6-861e-3bb948ca582c None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:52.973082 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15ef0f-d3b1-47a6-861e-3bb948ca582c None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:52.973484 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15ef0f-d3b1-47a6-861e-3bb948ca582c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:53.001820 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-fb15ef0f-d3b1-47a6-861e-3bb948ca582c None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:53.003212 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-fb15ef0f-d3b1-47a6-861e-3bb948ca582c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:53.010499 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac64c0f-b27a-4875-89a2-8eb4b074fd4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:53.012742 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 869/1753] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:53.016073 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac64c0f-b27a-4875-89a2-8eb4b074fd4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:53.016073 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac64c0f-b27a-4875-89a2-8eb4b074fd4f admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:53.016073 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac64c0f-b27a-4875-89a2-8eb4b074fd4f admin admin] PATH_INFO: `/v3/projects/a5fa23f308fd45f88add04f184ccd2b8` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:53.018547 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61494fbf-90cb-47b3-ac9f-b2a766887495 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:53.018984 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61494fbf-90cb-47b3-ac9f-b2a766887495 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:53.019415 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61494fbf-90cb-47b3-ac9f-b2a766887495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:53.021997 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac64c0f-b27a-4875-89a2-8eb4b074fd4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5fa23f308fd45f88add04f184ccd2b8)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:53.032825 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac64c0f-b27a-4875-89a2-8eb4b074fd4f admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:53.056913 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-61494fbf-90cb-47b3-ac9f-b2a766887495 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:53.058514 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-61494fbf-90cb-47b3-ac9f-b2a766887495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:53.070174 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 870/1754] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:53.071261 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61494fbf-90cb-47b3-ac9f-b2a766887495 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:53.077247 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-61494fbf-90cb-47b3-ac9f-b2a766887495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:53.114158 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-0ac64c0f-b27a-4875-89a2-8eb4b074fd4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5fa23f308fd45f88add04f184ccd2b8', 'request_id': 'req-0ac64c0f-b27a-4875-89a2-8eb4b074fd4f'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:53.135822 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-0ac64c0f-b27a-4875-89a2-8eb4b074fd4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5fa23f308fd45f88add04f184ccd2b8', 'request_id': 'req-0ac64c0f-b27a-4875-89a2-8eb4b074fd4f'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:53.155146 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-0ac64c0f-b27a-4875-89a2-8eb4b074fd4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5fa23f308fd45f88add04f184ccd2b8', 'request_id': 'req-0ac64c0f-b27a-4875-89a2-8eb4b074fd4f'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:53.181459 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-0ac64c0f-b27a-4875-89a2-8eb4b074fd4f admin admin] The token cache is being invalidate because project a5fa23f308fd45f88add04f184ccd2b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 14:58:53.181921 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-0ac64c0f-b27a-4875-89a2-8eb4b074fd4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:53.184022 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-0ac64c0f-b27a-4875-89a2-8eb4b074fd4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:53.187584 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61494fbf-90cb-47b3-ac9f-b2a766887495 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:53.189297 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 885/1755] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 14:58:52 2022] DELETE /identity/v3/projects/a5fa23f308fd45f88add04f184ccd2b8 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:53.190305 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac64c0f-b27a-4875-89a2-8eb4b074fd4f admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:53.194107 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-0ac64c0f-b27a-4875-89a2-8eb4b074fd4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:53.300939 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973fbfb7-2040-45ae-ac9f-f9abd2cd35e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:53.304867 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973fbfb7-2040-45ae-ac9f-f9abd2cd35e2 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:53.305175 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973fbfb7-2040-45ae-ac9f-f9abd2cd35e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:53.305466 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973fbfb7-2040-45ae-ac9f-f9abd2cd35e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:53.313254 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-973fbfb7-2040-45ae-ac9f-f9abd2cd35e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:53.313652 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac64c0f-b27a-4875-89a2-8eb4b074fd4f admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:53.319465 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e22dae-6833-4df9-b8a2-1c2708dd1f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:53.320191 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e22dae-6833-4df9-b8a2-1c2708dd1f10 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:53.320523 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e22dae-6833-4df9-b8a2-1c2708dd1f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:53.320882 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e22dae-6833-4df9-b8a2-1c2708dd1f10 admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:53.321754 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e22dae-6833-4df9-b8a2-1c2708dd1f10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:58:53.322279 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e22dae-6833-4df9-b8a2-1c2708dd1f10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:53.324182 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e22dae-6833-4df9-b8a2-1c2708dd1f10 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:53.327840 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973fbfb7-2040-45ae-ac9f-f9abd2cd35e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:53.330253 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973fbfb7-2040-45ae-ac9f-f9abd2cd35e2 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:53.341838 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 886/1756] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 14:58:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:58:53.342647 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 871/1757] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:58:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:58:53.344571 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530c13e-0fd4-4a09-b48f-3b3e4c2ee46b None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:53.344890 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530c13e-0fd4-4a09-b48f-3b3e4c2ee46b None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:53.345313 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530c13e-0fd4-4a09-b48f-3b3e4c2ee46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:53.348022 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973fbfb7-2040-45ae-ac9f-f9abd2cd35e2 service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:53.371673 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973fbfb7-2040-45ae-ac9f-f9abd2cd35e2 service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:53.374247 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-a530c13e-0fd4-4a09-b48f-3b3e4c2ee46b None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:53.375702 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-a530c13e-0fd4-4a09-b48f-3b3e4c2ee46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:53.390961 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9069de-57c5-4958-84c7-1e0ece2eba05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:53.391964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9069de-57c5-4958-84c7-1e0ece2eba05 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:53.392394 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9069de-57c5-4958-84c7-1e0ece2eba05 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:53.392903 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9069de-57c5-4958-84c7-1e0ece2eba05 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:53.393755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9069de-57c5-4958-84c7-1e0ece2eba05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:53.397100 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9069de-57c5-4958-84c7-1e0ece2eba05 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:53.400148 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 887/1758] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:53.412116 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d884c5-fd3a-4412-be5f-21fcd00df1ed None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:53.414310 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d884c5-fd3a-4412-be5f-21fcd00df1ed None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:53.414617 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d884c5-fd3a-4412-be5f-21fcd00df1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:53.451628 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-97d884c5-fd3a-4412-be5f-21fcd00df1ed None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:53.455820 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 872/1759] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:58:53 2022] POST /identity/v3/projects => generated 365 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:58:53.456354 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-97d884c5-fd3a-4412-be5f-21fcd00df1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:53.459223 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9069de-57c5-4958-84c7-1e0ece2eba05 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:53.563649 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 888/1760] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:53.564644 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d884c5-fd3a-4412-be5f-21fcd00df1ed None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:53.578705 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d884c5-fd3a-4412-be5f-21fcd00df1ed None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:53.594576 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d19c5bf-fdd4-4cbe-acf8-4b1fd3fd8e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:53.595184 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19c5bf-fdd4-4cbe-acf8-4b1fd3fd8e3d admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:53.595499 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19c5bf-fdd4-4cbe-acf8-4b1fd3fd8e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:53.595812 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19c5bf-fdd4-4cbe-acf8-4b1fd3fd8e3d admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:53.596682 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d19c5bf-fdd4-4cbe-acf8-4b1fd3fd8e3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:58:53.596999 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d19c5bf-fdd4-4cbe-acf8-4b1fd3fd8e3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:53.598625 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d19c5bf-fdd4-4cbe-acf8-4b1fd3fd8e3d admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:53.622022 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 889/1761] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 14:58:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:58:53.626897 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d19c5bf-fdd4-4cbe-acf8-4b1fd3fd8e3d admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:53.630253 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9069de-57c5-4958-84c7-1e0ece2eba05 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:53.635505 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3a48a0-bb39-44ff-8a06-0f5b8be5d10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:53.636194 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a48a0-bb39-44ff-8a06-0f5b8be5d10d admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:53.636485 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a48a0-bb39-44ff-8a06-0f5b8be5d10d admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:53.636934 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a48a0-bb39-44ff-8a06-0f5b8be5d10d admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:53.637827 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3a48a0-bb39-44ff-8a06-0f5b8be5d10d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:53.639831 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3a48a0-bb39-44ff-8a06-0f5b8be5d10d admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:53.641176 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d19c5bf-fdd4-4cbe-acf8-4b1fd3fd8e3d admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:53.647358 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc1c3e5-35ed-42ae-825c-205653c1d117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:53.648025 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc1c3e5-35ed-42ae-825c-205653c1d117 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:53.648311 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc1c3e5-35ed-42ae-825c-205653c1d117 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:53.648632 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc1c3e5-35ed-42ae-825c-205653c1d117 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:53.649508 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc1c3e5-35ed-42ae-825c-205653c1d117 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:53.652248 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc1c3e5-35ed-42ae-825c-205653c1d117 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:53.664190 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 873/1762] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:58:53 2022] POST /identity/v3/users => generated 283 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:58:53.669027 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3a48a0-bb39-44ff-8a06-0f5b8be5d10d admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:53.680039 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3a48a0-bb39-44ff-8a06-0f5b8be5d10d admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:53.683708 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 890/1763] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:58:53 2022] POST /identity/v3/projects => generated 365 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 14:58:53.685756 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f4467-cf28-4185-8b8e-a6aeebd60dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:53.686434 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f4467-cf28-4185-8b8e-a6aeebd60dfc admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:53.686718 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f4467-cf28-4185-8b8e-a6aeebd60dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:53.687316 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f4467-cf28-4185-8b8e-a6aeebd60dfc admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:53.688211 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602f4467-cf28-4185-8b8e-a6aeebd60dfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:53.688546 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc1c3e5-35ed-42ae-825c-205653c1d117 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:53.689857 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602f4467-cf28-4185-8b8e-a6aeebd60dfc admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:53.703574 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 874/1764] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 06 14:58:53.706255 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f4467-cf28-4185-8b8e-a6aeebd60dfc admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:53.718859 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f4467-cf28-4185-8b8e-a6aeebd60dfc admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:53.805387 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc1c3e5-35ed-42ae-825c-205653c1d117 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:53.813578 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c9b1a-2681-4551-9b21-129c0bed88e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:53.814301 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c9b1a-2681-4551-9b21-129c0bed88e8 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:53.814583 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c9b1a-2681-4551-9b21-129c0bed88e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:53.814859 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c9b1a-2681-4551-9b21-129c0bed88e8 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:53.815552 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c9b1a-2681-4551-9b21-129c0bed88e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:53.817694 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c9b1a-2681-4551-9b21-129c0bed88e8 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:53.835277 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3611603-4bd3-4b9f-be8d-56f68050a9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:53.836141 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3611603-4bd3-4b9f-be8d-56f68050a9ee admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:53.836566 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3611603-4bd3-4b9f-be8d-56f68050a9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:53.836858 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3611603-4bd3-4b9f-be8d-56f68050a9ee admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:53.838548 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3611603-4bd3-4b9f-be8d-56f68050a9ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:53.841694 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3611603-4bd3-4b9f-be8d-56f68050a9ee admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:53.858072 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 891/1765] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:58:53 2022] POST /identity/v3/users => generated 283 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:58:53.859890 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c9b1a-2681-4551-9b21-129c0bed88e8 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:53.865581 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 875/1766] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:58:53.866694 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3611603-4bd3-4b9f-be8d-56f68050a9ee admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:53.883052 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3611603-4bd3-4b9f-be8d-56f68050a9ee admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:53.888979 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f008de09-91ce-427a-b03a-7d076d1454c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:53.890781 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008de09-91ce-427a-b03a-7d076d1454c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:53.891078 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008de09-91ce-427a-b03a-7d076d1454c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:53.891354 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008de09-91ce-427a-b03a-7d076d1454c6 admin admin] PATH_INFO: `/v3/projects/d4f5a9c30c0d4ff18e3aef3314390f14/users/0ee92f0fe95843c697acc52b0a55bc16/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:53.908277 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f008de09-91ce-427a-b03a-7d076d1454c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4f5a9c30c0d4ff18e3aef3314390f14, user_id=0ee92f0fe95843c697acc52b0a55bc16, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:53.909163 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c9b1a-2681-4551-9b21-129c0bed88e8 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:53.911649 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f008de09-91ce-427a-b03a-7d076d1454c6 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:53.922053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f82e851-99b6-42fe-8952-43e246518bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:53.923772 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f82e851-99b6-42fe-8952-43e246518bbe admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:53.925168 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f82e851-99b6-42fe-8952-43e246518bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:53.925685 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f82e851-99b6-42fe-8952-43e246518bbe admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:53.927541 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f82e851-99b6-42fe-8952-43e246518bbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:53.936642 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 876/1767] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:58:53 2022] PUT /identity/v3/projects/d4f5a9c30c0d4ff18e3aef3314390f14/users/0ee92f0fe95843c697acc52b0a55bc16/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:53.937310 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f82e851-99b6-42fe-8952-43e246518bbe admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:53.941477 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae985c-62da-4333-a193-9bfca3a75bf5 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:53.942072 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae985c-62da-4333-a193-9bfca3a75bf5 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:53.942663 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae985c-62da-4333-a193-9bfca3a75bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:53.955133 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 892/1768] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 06 14:58:53.956417 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f82e851-99b6-42fe-8952-43e246518bbe admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:53.958533 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-7f82e851-99b6-42fe-8952-43e246518bbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:53.989039 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f82e851-99b6-42fe-8952-43e246518bbe admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:53.990089 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-39ae985c-62da-4333-a193-9bfca3a75bf5 None None] MFA Rules not processed for user `0ee92f0fe95843c697acc52b0a55bc16`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:53.999598 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-39ae985c-62da-4333-a193-9bfca3a75bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:54.127173 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd46a33-b9e8-4980-ac6b-b9965105b02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:54.128160 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd46a33-b9e8-4980-ac6b-b9965105b02a admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:54.129874 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd46a33-b9e8-4980-ac6b-b9965105b02a admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:54.129874 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd46a33-b9e8-4980-ac6b-b9965105b02a admin admin] PATH_INFO: `/v3/projects/1f5179c8b74b424092d8e3f6ec37316f` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:54.131154 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd46a33-b9e8-4980-ac6b-b9965105b02a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f5179c8b74b424092d8e3f6ec37316f)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:54.131891 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 877/1769] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:53 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 14:58:54.132765 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ae985c-62da-4333-a193-9bfca3a75bf5 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:54.135749 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd46a33-b9e8-4980-ac6b-b9965105b02a admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:54.145866 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ae985c-62da-4333-a193-9bfca3a75bf5 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:54.151553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59080174-e2ce-4caf-b8ad-5b2a0d9e8b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:54.152288 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59080174-e2ce-4caf-b8ad-5b2a0d9e8b2f admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:54.152636 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59080174-e2ce-4caf-b8ad-5b2a0d9e8b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:54.152965 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59080174-e2ce-4caf-b8ad-5b2a0d9e8b2f admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:54.153903 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59080174-e2ce-4caf-b8ad-5b2a0d9e8b2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:54.155486 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59080174-e2ce-4caf-b8ad-5b2a0d9e8b2f admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:54.168786 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 878/1770] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:58:54.168786 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59080174-e2ce-4caf-b8ad-5b2a0d9e8b2f admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:54.179097 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59080174-e2ce-4caf-b8ad-5b2a0d9e8b2f admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:54.195867 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-edd46a33-b9e8-4980-ac6b-b9965105b02a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f5179c8b74b424092d8e3f6ec37316f', 'request_id': 'req-edd46a33-b9e8-4980-ac6b-b9965105b02a'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:54.202302 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-edd46a33-b9e8-4980-ac6b-b9965105b02a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f5179c8b74b424092d8e3f6ec37316f', 'request_id': 'req-edd46a33-b9e8-4980-ac6b-b9965105b02a'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:54.210983 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-edd46a33-b9e8-4980-ac6b-b9965105b02a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f5179c8b74b424092d8e3f6ec37316f', 'request_id': 'req-edd46a33-b9e8-4980-ac6b-b9965105b02a'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:54.224760 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-edd46a33-b9e8-4980-ac6b-b9965105b02a admin admin] The token cache is being invalidate because project 1f5179c8b74b424092d8e3f6ec37316f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 14:58:54.225097 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-edd46a33-b9e8-4980-ac6b-b9965105b02a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:54.225779 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-edd46a33-b9e8-4980-ac6b-b9965105b02a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:54.227337 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 893/1771] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 14:58:53 2022] DELETE /identity/v3/projects/1f5179c8b74b424092d8e3f6ec37316f => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:54.228156 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd46a33-b9e8-4980-ac6b-b9965105b02a admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:54.229513 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-edd46a33-b9e8-4980-ac6b-b9965105b02a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:54.246110 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36882f6f-d63a-4c99-a90f-e10edb01b24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:54.246778 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36882f6f-d63a-4c99-a90f-e10edb01b24c admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:54.247057 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36882f6f-d63a-4c99-a90f-e10edb01b24c admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:54.247324 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36882f6f-d63a-4c99-a90f-e10edb01b24c admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:54.248008 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36882f6f-d63a-4c99-a90f-e10edb01b24c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:54.249955 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36882f6f-d63a-4c99-a90f-e10edb01b24c admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:54.276527 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 879/1772] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:58:54 2022] POST /identity/v3/projects => generated 365 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:58:54.277466 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36882f6f-d63a-4c99-a90f-e10edb01b24c admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:54.279214 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-36882f6f-d63a-4c99-a90f-e10edb01b24c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:54.308774 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd46a33-b9e8-4980-ac6b-b9965105b02a admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:54.359763 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36882f6f-d63a-4c99-a90f-e10edb01b24c admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:54.364829 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24412722-bb55-47b5-ac03-82f9909b60e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:54.365711 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24412722-bb55-47b5-ac03-82f9909b60e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:54.365993 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24412722-bb55-47b5-ac03-82f9909b60e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:54.366258 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24412722-bb55-47b5-ac03-82f9909b60e8 admin admin] PATH_INFO: `/v3/users/7178c98d418f4b19a209dcdbf1ffa921` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:54.367657 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24412722-bb55-47b5-ac03-82f9909b60e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7178c98d418f4b19a209dcdbf1ffa921)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:54.370320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24412722-bb55-47b5-ac03-82f9909b60e8 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:54.378867 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46569b38-354c-4a50-95b9-426f6070d121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:54.379957 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46569b38-354c-4a50-95b9-426f6070d121 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:54.380205 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46569b38-354c-4a50-95b9-426f6070d121 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:54.380481 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46569b38-354c-4a50-95b9-426f6070d121 admin admin] PATH_INFO: `/v3/projects/ca1c4a09ef394697bbc8317373a4c547/users/6db19b0474194200a90bd9d6356b8039/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:54.397074 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46569b38-354c-4a50-95b9-426f6070d121 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca1c4a09ef394697bbc8317373a4c547, user_id=6db19b0474194200a90bd9d6356b8039, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:54.400217 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46569b38-354c-4a50-95b9-426f6070d121 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:54.420701 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 894/1773] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:58:54 2022] PUT /identity/v3/projects/ca1c4a09ef394697bbc8317373a4c547/users/6db19b0474194200a90bd9d6356b8039/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:54.425079 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46569b38-354c-4a50-95b9-426f6070d121 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:54.426442 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-46569b38-354c-4a50-95b9-426f6070d121 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:54.444887 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-24412722-bb55-47b5-ac03-82f9909b60e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7178c98d418f4b19a209dcdbf1ffa921'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:54.445570 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-24412722-bb55-47b5-ac03-82f9909b60e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7178c98d418f4b19a209dcdbf1ffa921'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:54.454871 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-24412722-bb55-47b5-ac03-82f9909b60e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7178c98d418f4b19a209dcdbf1ffa921'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:54.462641 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-24412722-bb55-47b5-ac03-82f9909b60e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7178c98d418f4b19a209dcdbf1ffa921'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:54.463688 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-24412722-bb55-47b5-ac03-82f9909b60e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7178c98d418f4b19a209dcdbf1ffa921'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:54.483342 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 880/1774] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 14:58:54 2022] DELETE /identity/v3/users/7178c98d418f4b19a209dcdbf1ffa921 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:54.484805 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a11496-de69-4d9a-8f11-ef3046dd9e93 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:54.485065 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a11496-de69-4d9a-8f11-ef3046dd9e93 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:54.485343 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a11496-de69-4d9a-8f11-ef3046dd9e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:54.500052 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46569b38-354c-4a50-95b9-426f6070d121 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:54.514520 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-27a11496-de69-4d9a-8f11-ef3046dd9e93 None None] MFA Rules not processed for user `6db19b0474194200a90bd9d6356b8039`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:54.515936 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-27a11496-de69-4d9a-8f11-ef3046dd9e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:54.606740 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 881/1775] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:54 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:54.611846 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c965a7-5406-4118-b7fc-f7ff564d0354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:54.611846 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c965a7-5406-4118-b7fc-f7ff564d0354 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:54.612085 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a11496-de69-4d9a-8f11-ef3046dd9e93 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:54.612783 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c965a7-5406-4118-b7fc-f7ff564d0354 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:54.613318 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-27a11496-de69-4d9a-8f11-ef3046dd9e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:54.613746 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c965a7-5406-4118-b7fc-f7ff564d0354 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:54.614997 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c965a7-5406-4118-b7fc-f7ff564d0354 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:54.617442 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c965a7-5406-4118-b7fc-f7ff564d0354 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:54.638836 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a11496-de69-4d9a-8f11-ef3046dd9e93 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:54.644353 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9eff26-cbdb-4853-a93c-b32c6ec80bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:54.645374 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9eff26-cbdb-4853-a93c-b32c6ec80bb6 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:54.645664 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9eff26-cbdb-4853-a93c-b32c6ec80bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:54.646131 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9eff26-cbdb-4853-a93c-b32c6ec80bb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:54.646849 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9eff26-cbdb-4853-a93c-b32c6ec80bb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:54.650344 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9eff26-cbdb-4853-a93c-b32c6ec80bb6 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:54.650905 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 895/1776] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:58:54 2022] POST /identity/v3/users => generated 289 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:58:54.654844 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c965a7-5406-4118-b7fc-f7ff564d0354 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:54.657776 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e8c965a7-5406-4118-b7fc-f7ff564d0354 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:54.683293 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c965a7-5406-4118-b7fc-f7ff564d0354 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:54.688402 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407c68f6-96ba-448f-ab41-2f045353ad45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:54.689128 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c68f6-96ba-448f-ab41-2f045353ad45 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:54.689412 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c68f6-96ba-448f-ab41-2f045353ad45 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:54.689684 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c68f6-96ba-448f-ab41-2f045353ad45 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:54.690529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407c68f6-96ba-448f-ab41-2f045353ad45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:54.692008 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407c68f6-96ba-448f-ab41-2f045353ad45 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:54.693325 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 882/1777] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:58:54 2022] POST /identity/v3/projects => generated 365 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 14:58:54.698465 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9eff26-cbdb-4853-a93c-b32c6ec80bb6 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:54.730172 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 896/1778] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:54 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:58:54.731318 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407c68f6-96ba-448f-ab41-2f045353ad45 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:54.743583 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407c68f6-96ba-448f-ab41-2f045353ad45 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:54.794334 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9eff26-cbdb-4853-a93c-b32c6ec80bb6 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:54.800517 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85716a16-fdfd-4c75-aee2-0acf34b86672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:54.801406 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85716a16-fdfd-4c75-aee2-0acf34b86672 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:54.802273 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85716a16-fdfd-4c75-aee2-0acf34b86672 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:54.802273 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85716a16-fdfd-4c75-aee2-0acf34b86672 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:54.803088 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85716a16-fdfd-4c75-aee2-0acf34b86672 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:54.805306 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85716a16-fdfd-4c75-aee2-0acf34b86672 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:54.820346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39128fc-b79a-4812-af13-e17773f6a647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:54.821391 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39128fc-b79a-4812-af13-e17773f6a647 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:54.822324 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39128fc-b79a-4812-af13-e17773f6a647 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:54.822671 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39128fc-b79a-4812-af13-e17773f6a647 admin admin] PATH_INFO: `/v3/projects/952f23964070490ca0912354e4a1ef64/users/327b46b84403423d8a01218c9700c9d2/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:54.836228 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 883/1779] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:58:54 2022] POST /identity/v3/users => generated 289 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:58:54.854082 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85716a16-fdfd-4c75-aee2-0acf34b86672 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:54.863987 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39128fc-b79a-4812-af13-e17773f6a647 admin admin] RBAC: Authorizing `identity:create_grant(project_id=952f23964070490ca0912354e4a1ef64, user_id=327b46b84403423d8a01218c9700c9d2, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:54.869283 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39128fc-b79a-4812-af13-e17773f6a647 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:54.877628 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85716a16-fdfd-4c75-aee2-0acf34b86672 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:54.884145 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbdd523-6f41-4741-9336-52dbf5f5497f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:54.886402 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdd523-6f41-4741-9336-52dbf5f5497f admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:54.886743 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdd523-6f41-4741-9336-52dbf5f5497f admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:54.887026 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdd523-6f41-4741-9336-52dbf5f5497f admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:54.887768 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbdd523-6f41-4741-9336-52dbf5f5497f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:54.889692 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbdd523-6f41-4741-9336-52dbf5f5497f admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:54.891485 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 897/1780] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:58:54 2022] PUT /identity/v3/projects/952f23964070490ca0912354e4a1ef64/users/327b46b84403423d8a01218c9700c9d2/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:54.894717 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39128fc-b79a-4812-af13-e17773f6a647 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:54.896593 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-c39128fc-b79a-4812-af13-e17773f6a647 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:54.908666 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 884/1781] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:54 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:58:54.908666 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbdd523-6f41-4741-9336-52dbf5f5497f admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:54.922204 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbdd523-6f41-4741-9336-52dbf5f5497f admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:55.016147 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39128fc-b79a-4812-af13-e17773f6a647 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:55.023051 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91537e3-e77c-4770-a319-3bd4645f068c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:55.023931 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91537e3-e77c-4770-a319-3bd4645f068c admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:55.024214 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91537e3-e77c-4770-a319-3bd4645f068c admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:55.024495 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91537e3-e77c-4770-a319-3bd4645f068c admin admin] PATH_INFO: `/v3/projects/d3f04cd2eb134745ae384a0e64cd03e0` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:55.027168 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91537e3-e77c-4770-a319-3bd4645f068c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3f04cd2eb134745ae384a0e64cd03e0)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:55.029817 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91537e3-e77c-4770-a319-3bd4645f068c admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:55.105165 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c2a1ad-4bea-46ef-8be0-4f399f417ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:55.105850 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c2a1ad-4bea-46ef-8be0-4f399f417ae2 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:55.106142 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c2a1ad-4bea-46ef-8be0-4f399f417ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:55.106400 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c2a1ad-4bea-46ef-8be0-4f399f417ae2 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:55.107188 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c2a1ad-4bea-46ef-8be0-4f399f417ae2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:55.109304 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c2a1ad-4bea-46ef-8be0-4f399f417ae2 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:55.112905 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-e91537e3-e77c-4770-a319-3bd4645f068c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3f04cd2eb134745ae384a0e64cd03e0', 'request_id': 'req-e91537e3-e77c-4770-a319-3bd4645f068c'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:55.124207 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 885/1782] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:54 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:58:55.129039 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c2a1ad-4bea-46ef-8be0-4f399f417ae2 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:55.130071 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-e91537e3-e77c-4770-a319-3bd4645f068c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3f04cd2eb134745ae384a0e64cd03e0', 'request_id': 'req-e91537e3-e77c-4770-a319-3bd4645f068c'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:55.158590 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c2a1ad-4bea-46ef-8be0-4f399f417ae2 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:55.163562 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-e91537e3-e77c-4770-a319-3bd4645f068c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3f04cd2eb134745ae384a0e64cd03e0', 'request_id': 'req-e91537e3-e77c-4770-a319-3bd4645f068c'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:55.205184 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-e91537e3-e77c-4770-a319-3bd4645f068c admin admin] The token cache is being invalidate because project d3f04cd2eb134745ae384a0e64cd03e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 14:58:55.205585 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-e91537e3-e77c-4770-a319-3bd4645f068c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:55.217299 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-e91537e3-e77c-4770-a319-3bd4645f068c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:55.227080 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 898/1783] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 14:58:54 2022] DELETE /identity/v3/projects/d3f04cd2eb134745ae384a0e64cd03e0 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:55.228259 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91537e3-e77c-4770-a319-3bd4645f068c admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:55.234484 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e91537e3-e77c-4770-a319-3bd4645f068c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:55.360283 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62062e68-1e03-414b-82d6-cba3d7d1b7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:55.361587 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62062e68-1e03-414b-82d6-cba3d7d1b7ca admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:55.362079 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62062e68-1e03-414b-82d6-cba3d7d1b7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:55.362553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62062e68-1e03-414b-82d6-cba3d7d1b7ca admin admin] PATH_INFO: `/v3/projects/30a08c313b4745bd9e2e9a5dfe19672b/users/08c621d05f5748cf8b7d1eb9f03d7f1e/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:55.393002 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62062e68-1e03-414b-82d6-cba3d7d1b7ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=30a08c313b4745bd9e2e9a5dfe19672b, user_id=08c621d05f5748cf8b7d1eb9f03d7f1e, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:55.397494 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62062e68-1e03-414b-82d6-cba3d7d1b7ca admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:55.419609 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91537e3-e77c-4770-a319-3bd4645f068c admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:55.425321 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 886/1784] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:58:55 2022] PUT /identity/v3/projects/30a08c313b4745bd9e2e9a5dfe19672b/users/08c621d05f5748cf8b7d1eb9f03d7f1e/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:55.428716 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac091cf-0171-4723-904d-c5dc909cad1b None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:55.430100 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac091cf-0171-4723-904d-c5dc909cad1b None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:55.430398 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac091cf-0171-4723-904d-c5dc909cad1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:55.471143 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-bac091cf-0171-4723-904d-c5dc909cad1b None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:55.472530 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-bac091cf-0171-4723-904d-c5dc909cad1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:55.495850 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b35e5f0-162f-4ef0-afb0-bdb6fab97c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:55.496951 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35e5f0-162f-4ef0-afb0-bdb6fab97c50 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:55.497216 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35e5f0-162f-4ef0-afb0-bdb6fab97c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:55.497942 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35e5f0-162f-4ef0-afb0-bdb6fab97c50 admin admin] PATH_INFO: `/v3/projects/952f23964070490ca0912354e4a1ef64/users/327b46b84403423d8a01218c9700c9d2/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:55.500723 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b35e5f0-162f-4ef0-afb0-bdb6fab97c50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=952f23964070490ca0912354e4a1ef64, user_id=327b46b84403423d8a01218c9700c9d2, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:55.503660 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b35e5f0-162f-4ef0-afb0-bdb6fab97c50 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:55.532765 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 899/1785] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:58:55 2022] PUT /identity/v3/projects/952f23964070490ca0912354e4a1ef64/users/327b46b84403423d8a01218c9700c9d2/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:55.542835 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b35e5f0-162f-4ef0-afb0-bdb6fab97c50 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:55.545773 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-4b35e5f0-162f-4ef0-afb0-bdb6fab97c50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:55.594918 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 887/1786] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:55.596579 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6337ca-4e02-49a6-8457-ac54a2ed79c3 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:55.597099 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6337ca-4e02-49a6-8457-ac54a2ed79c3 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:55.597452 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6337ca-4e02-49a6-8457-ac54a2ed79c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:55.645438 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-ed6337ca-4e02-49a6-8457-ac54a2ed79c3 None None] MFA Rules not processed for user `327b46b84403423d8a01218c9700c9d2`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:55.647938 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ed6337ca-4e02-49a6-8457-ac54a2ed79c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:55.686345 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b35e5f0-162f-4ef0-afb0-bdb6fab97c50 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:55.699601 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779c76ea-965a-4a2f-9365-9c96324af87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:55.700308 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c76ea-965a-4a2f-9365-9c96324af87d admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:55.700620 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c76ea-965a-4a2f-9365-9c96324af87d admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:55.700923 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c76ea-965a-4a2f-9365-9c96324af87d admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:55.701669 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779c76ea-965a-4a2f-9365-9c96324af87d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:55.703506 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779c76ea-965a-4a2f-9365-9c96324af87d admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:55.721867 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 900/1787] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:58:55.723212 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02fc508-baf8-4f13-a8b7-6d84c1b7ac54 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:55.724508 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02fc508-baf8-4f13-a8b7-6d84c1b7ac54 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:55.724994 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02fc508-baf8-4f13-a8b7-6d84c1b7ac54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:55.752233 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-b02fc508-baf8-4f13-a8b7-6d84c1b7ac54 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:55.753611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-b02fc508-baf8-4f13-a8b7-6d84c1b7ac54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:55.774284 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 901/1788] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:55.775499 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02fc508-baf8-4f13-a8b7-6d84c1b7ac54 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:55.780207 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 888/1789] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:55 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:55.781984 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6337ca-4e02-49a6-8457-ac54a2ed79c3 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:55.795581 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6337ca-4e02-49a6-8457-ac54a2ed79c3 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:55.801153 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02fc508-baf8-4f13-a8b7-6d84c1b7ac54 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:55.804167 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e39952f-7036-4b25-9742-4de5943ead27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:55.805050 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39952f-7036-4b25-9742-4de5943ead27 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:55.805332 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39952f-7036-4b25-9742-4de5943ead27 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:55.805664 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39952f-7036-4b25-9742-4de5943ead27 admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:55.807513 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e39952f-7036-4b25-9742-4de5943ead27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:58:55.808257 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e39952f-7036-4b25-9742-4de5943ead27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:55.808977 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599eca12-c3a9-464d-9990-fceb8a37ff82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:55.809458 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e39952f-7036-4b25-9742-4de5943ead27 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:55.810306 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599eca12-c3a9-464d-9990-fceb8a37ff82 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:55.810802 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599eca12-c3a9-464d-9990-fceb8a37ff82 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:55.811617 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599eca12-c3a9-464d-9990-fceb8a37ff82 admin admin] PATH_INFO: `/v3/projects/30a08c313b4745bd9e2e9a5dfe19672b/users/08c621d05f5748cf8b7d1eb9f03d7f1e/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:55.814452 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599eca12-c3a9-464d-9990-fceb8a37ff82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30a08c313b4745bd9e2e9a5dfe19672b, user_id=08c621d05f5748cf8b7d1eb9f03d7f1e, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:55.818654 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599eca12-c3a9-464d-9990-fceb8a37ff82 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:55.836664 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 889/1790] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 14:58:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:58:55.838470 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e39952f-7036-4b25-9742-4de5943ead27 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:55.840786 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-3e39952f-7036-4b25-9742-4de5943ead27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:55.842133 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 902/1791] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:58:55 2022] PUT /identity/v3/projects/30a08c313b4745bd9e2e9a5dfe19672b/users/08c621d05f5748cf8b7d1eb9f03d7f1e/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:55.843148 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599eca12-c3a9-464d-9990-fceb8a37ff82 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:55.920388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e39952f-7036-4b25-9742-4de5943ead27 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:55.930900 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599eca12-c3a9-464d-9990-fceb8a37ff82 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:55.936339 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e2257-28e3-416e-9b3c-a818421deac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:55.937070 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e2257-28e3-416e-9b3c-a818421deac5 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:55.937350 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e2257-28e3-416e-9b3c-a818421deac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:55.937622 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e2257-28e3-416e-9b3c-a818421deac5 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:55.938302 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304e2257-28e3-416e-9b3c-a818421deac5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:55.940177 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304e2257-28e3-416e-9b3c-a818421deac5 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:55.975778 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 903/1792] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:58:55 2022] POST /identity/v3/projects => generated 383 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:58:55.978131 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084e0b4-998c-4baa-947a-142900e591ed None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:55.978593 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084e0b4-998c-4baa-947a-142900e591ed None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:55.979043 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084e0b4-998c-4baa-947a-142900e591ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:55.987466 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe58bc6-61e1-4715-a0c9-aa8f7f1357f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:55.988263 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe58bc6-61e1-4715-a0c9-aa8f7f1357f5 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:55.988677 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe58bc6-61e1-4715-a0c9-aa8f7f1357f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:55.989067 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe58bc6-61e1-4715-a0c9-aa8f7f1357f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:56.018887 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-9084e0b4-998c-4baa-947a-142900e591ed None None] MFA Rules not processed for user `08c621d05f5748cf8b7d1eb9f03d7f1e`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:56.020743 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-9084e0b4-998c-4baa-947a-142900e591ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:56.069667 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe58bc6-61e1-4715-a0c9-aa8f7f1357f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:56.072942 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe58bc6-61e1-4715-a0c9-aa8f7f1357f5 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:56.090920 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 890/1793] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:58:55 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:58:56.091760 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe58bc6-61e1-4715-a0c9-aa8f7f1357f5 service cinder] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:56.101780 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 904/1794] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:55 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:56.107711 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe58bc6-61e1-4715-a0c9-aa8f7f1357f5 service cinder] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:56.114357 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9084e0b4-998c-4baa-947a-142900e591ed None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:56.115724 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-9084e0b4-998c-4baa-947a-142900e591ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:56.172439 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62155cab-23fb-436d-82b0-7dff51ce4b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:56.173144 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62155cab-23fb-436d-82b0-7dff51ce4b82 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:56.173425 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62155cab-23fb-436d-82b0-7dff51ce4b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:56.173693 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62155cab-23fb-436d-82b0-7dff51ce4b82 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:56.174632 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62155cab-23fb-436d-82b0-7dff51ce4b82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:56.176358 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62155cab-23fb-436d-82b0-7dff51ce4b82 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:56.186137 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9084e0b4-998c-4baa-947a-142900e591ed None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:56.190551 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483db2e1-ca3a-4f6b-880c-2e6f4b8a9f3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:56.191469 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483db2e1-ca3a-4f6b-880c-2e6f4b8a9f3d service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:56.191757 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483db2e1-ca3a-4f6b-880c-2e6f4b8a9f3d service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:56.192026 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483db2e1-ca3a-4f6b-880c-2e6f4b8a9f3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:56.196119 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 891/1795] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:58:56 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:58:56.197577 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62155cab-23fb-436d-82b0-7dff51ce4b82 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:56.202488 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483db2e1-ca3a-4f6b-880c-2e6f4b8a9f3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:56.204010 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483db2e1-ca3a-4f6b-880c-2e6f4b8a9f3d service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:56.210462 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62155cab-23fb-436d-82b0-7dff51ce4b82 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:56.214871 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 905/1796] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:58:56 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 14:58:56.215664 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483db2e1-ca3a-4f6b-880c-2e6f4b8a9f3d service cinder] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:56.217027 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5669dbc1-a591-4a56-beae-5879211ad173 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:56.217871 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669dbc1-a591-4a56-beae-5879211ad173 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:56.218256 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669dbc1-a591-4a56-beae-5879211ad173 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:56.218634 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669dbc1-a591-4a56-beae-5879211ad173 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:56.220710 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-5669dbc1-a591-4a56-beae-5879211ad173 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:56.225338 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483db2e1-ca3a-4f6b-880c-2e6f4b8a9f3d service cinder] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:56.230392 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e54dd5b-4d21-45d9-b94a-d212472d7413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:56.231052 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54dd5b-4d21-45d9-b94a-d212472d7413 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:56.231332 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54dd5b-4d21-45d9-b94a-d212472d7413 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:56.231602 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54dd5b-4d21-45d9-b94a-d212472d7413 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:56.232721 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e54dd5b-4d21-45d9-b94a-d212472d7413 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:56.234237 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e54dd5b-4d21-45d9-b94a-d212472d7413 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:56.246016 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 906/1797] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:58:56.250195 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e54dd5b-4d21-45d9-b94a-d212472d7413 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:56.267649 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e54dd5b-4d21-45d9-b94a-d212472d7413 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:56.273451 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd2eafc-5c05-4d12-ad1b-b54dd244e637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:56.274203 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2eafc-5c05-4d12-ad1b-b54dd244e637 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:56.274820 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2eafc-5c05-4d12-ad1b-b54dd244e637 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:56.275666 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2eafc-5c05-4d12-ad1b-b54dd244e637 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:56.276822 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd2eafc-5c05-4d12-ad1b-b54dd244e637 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:56.278461 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd2eafc-5c05-4d12-ad1b-b54dd244e637 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:56.289877 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 907/1798] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:58:56.294030 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd2eafc-5c05-4d12-ad1b-b54dd244e637 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:56.294979 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5669dbc1-a591-4a56-beae-5879211ad173 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:56.296425 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5669dbc1-a591-4a56-beae-5879211ad173 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:56.304973 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd2eafc-5c05-4d12-ad1b-b54dd244e637 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:56.306210 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 892/1799] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:58:56 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:58:56.310517 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4645a4a-62bd-4173-9a6e-a29786159e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:56.311358 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4645a4a-62bd-4173-9a6e-a29786159e49 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:56.311638 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4645a4a-62bd-4173-9a6e-a29786159e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:56.311912 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4645a4a-62bd-4173-9a6e-a29786159e49 admin admin] PATH_INFO: `/v3/projects/f96882caf2fb4424bf4bd786bbd56af0/users/84713df215394af981221803bc03cb7c/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:56.328334 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4645a4a-62bd-4173-9a6e-a29786159e49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f96882caf2fb4424bf4bd786bbd56af0, user_id=84713df215394af981221803bc03cb7c, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:56.331010 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4645a4a-62bd-4173-9a6e-a29786159e49 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:56.349006 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 908/1800] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:58:56 2022] PUT /identity/v3/projects/f96882caf2fb4424bf4bd786bbd56af0/users/84713df215394af981221803bc03cb7c/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 14:58:56.360327 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002a17b-7b4d-4ef1-9b3c-c5b17584555d None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:56.360651 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002a17b-7b4d-4ef1-9b3c-c5b17584555d None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:56.360934 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002a17b-7b4d-4ef1-9b3c-c5b17584555d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:56.408289 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-7002a17b-7b4d-4ef1-9b3c-c5b17584555d None None] MFA Rules not processed for user `84713df215394af981221803bc03cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:56.410860 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-7002a17b-7b4d-4ef1-9b3c-c5b17584555d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:56.514166 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 893/1801] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:56.521203 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4645a4a-62bd-4173-9a6e-a29786159e49 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:56.630109 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4645a4a-62bd-4173-9a6e-a29786159e49 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:56.636879 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c9aab0-e92f-4da4-9da9-3911cc60476c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:56.637605 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c9aab0-e92f-4da4-9da9-3911cc60476c admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:56.637885 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c9aab0-e92f-4da4-9da9-3911cc60476c admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:56.638161 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c9aab0-e92f-4da4-9da9-3911cc60476c admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:56.638856 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c9aab0-e92f-4da4-9da9-3911cc60476c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:56.640955 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c9aab0-e92f-4da4-9da9-3911cc60476c admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:56.679031 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 909/1802] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:58:56 2022] POST /identity/v3/projects => generated 381 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 14:58:56.683828 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7002a17b-7b4d-4ef1-9b3c-c5b17584555d None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:56.697139 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7002a17b-7b4d-4ef1-9b3c-c5b17584555d None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:56.772011 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a783a08b-ce49-4bde-a0f4-43b876ebb421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:56.775293 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a783a08b-ce49-4bde-a0f4-43b876ebb421 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:56.775944 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a783a08b-ce49-4bde-a0f4-43b876ebb421 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:56.776788 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a783a08b-ce49-4bde-a0f4-43b876ebb421 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:56.779836 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a783a08b-ce49-4bde-a0f4-43b876ebb421 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:56.782939 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a783a08b-ce49-4bde-a0f4-43b876ebb421 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:56.802529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c9aab0-e92f-4da4-9da9-3911cc60476c admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:56.815745 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c9aab0-e92f-4da4-9da9-3911cc60476c admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:56.818296 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 894/1803] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:58:56 2022] POST /identity/v3/users => generated 297 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 14:58:56.823356 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a783a08b-ce49-4bde-a0f4-43b876ebb421 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:56.827569 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab48e7d-e5d4-4788-941b-880a58f68d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:56.830606 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab48e7d-e5d4-4788-941b-880a58f68d2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:56.831031 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab48e7d-e5d4-4788-941b-880a58f68d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:56.834352 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab48e7d-e5d4-4788-941b-880a58f68d2b admin admin] PATH_INFO: `/v3/users/6db19b0474194200a90bd9d6356b8039` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:56.835483 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab48e7d-e5d4-4788-941b-880a58f68d2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6db19b0474194200a90bd9d6356b8039)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:56.838685 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab48e7d-e5d4-4788-941b-880a58f68d2b admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:56.852245 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a783a08b-ce49-4bde-a0f4-43b876ebb421 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:56.861130 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d48fb93-607a-49c8-9e63-c516341c0b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:56.861863 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d48fb93-607a-49c8-9e63-c516341c0b32 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:56.862145 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d48fb93-607a-49c8-9e63-c516341c0b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:56.862472 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d48fb93-607a-49c8-9e63-c516341c0b32 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:56.863202 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d48fb93-607a-49c8-9e63-c516341c0b32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:56.865177 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d48fb93-607a-49c8-9e63-c516341c0b32 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:56.940390 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 895/1804] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:58:56.943294 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d48fb93-607a-49c8-9e63-c516341c0b32 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:56.972930 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d48fb93-607a-49c8-9e63-c516341c0b32 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:56.987777 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea1005-0371-410d-bd42-12f304b4fdab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:56.989167 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea1005-0371-410d-bd42-12f304b4fdab admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:56.989456 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea1005-0371-410d-bd42-12f304b4fdab admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:56.989730 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea1005-0371-410d-bd42-12f304b4fdab admin admin] PATH_INFO: `/v3/projects/6ee68b53c65d4c4c8fc909b35095e8d6/users/4bd987b4f83140f9be828ec04809a16c/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:56.992139 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-cab48e7d-e5d4-4788-941b-880a58f68d2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6db19b0474194200a90bd9d6356b8039'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:57.009882 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-cab48e7d-e5d4-4788-941b-880a58f68d2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6db19b0474194200a90bd9d6356b8039'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:57.022607 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-cab48e7d-e5d4-4788-941b-880a58f68d2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6db19b0474194200a90bd9d6356b8039'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:57.024111 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-cab48e7d-e5d4-4788-941b-880a58f68d2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6db19b0474194200a90bd9d6356b8039'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:57.027029 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-cab48e7d-e5d4-4788-941b-880a58f68d2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6db19b0474194200a90bd9d6356b8039'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:57.033608 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dea1005-0371-410d-bd42-12f304b4fdab admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ee68b53c65d4c4c8fc909b35095e8d6, user_id=4bd987b4f83140f9be828ec04809a16c, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:57.037605 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dea1005-0371-410d-bd42-12f304b4fdab admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:57.060642 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 896/1805] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:58:56 2022] PUT /identity/v3/projects/6ee68b53c65d4c4c8fc909b35095e8d6/users/4bd987b4f83140f9be828ec04809a16c/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:57.063109 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea1005-0371-410d-bd42-12f304b4fdab admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:57.065257 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-6dea1005-0371-410d-bd42-12f304b4fdab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:57.068228 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 910/1806] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 14:58:56 2022] DELETE /identity/v3/users/6db19b0474194200a90bd9d6356b8039 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:57.073480 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345688d9-9cb5-497a-8223-cba5c43951f2 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:57.073989 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345688d9-9cb5-497a-8223-cba5c43951f2 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:57.074363 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345688d9-9cb5-497a-8223-cba5c43951f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:57.115697 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-345688d9-9cb5-497a-8223-cba5c43951f2 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:57.119362 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-345688d9-9cb5-497a-8223-cba5c43951f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:57.172816 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea1005-0371-410d-bd42-12f304b4fdab admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:57.251470 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595e800-d42c-4515-bdf3-40e4c768ffbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:57.252476 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595e800-d42c-4515-bdf3-40e4c768ffbf admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:57.253604 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595e800-d42c-4515-bdf3-40e4c768ffbf admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:57.254088 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595e800-d42c-4515-bdf3-40e4c768ffbf admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:57.255670 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5595e800-d42c-4515-bdf3-40e4c768ffbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:57.258091 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5595e800-d42c-4515-bdf3-40e4c768ffbf admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:57.291327 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 911/1807] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:57.297511 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 897/1808] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:58:57.298510 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345688d9-9cb5-497a-8223-cba5c43951f2 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:57.312899 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345688d9-9cb5-497a-8223-cba5c43951f2 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:57.318110 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71650274-eef3-4fff-bd79-de5c492eee5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:57.319023 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71650274-eef3-4fff-bd79-de5c492eee5f admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:57.319306 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71650274-eef3-4fff-bd79-de5c492eee5f admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:57.319585 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71650274-eef3-4fff-bd79-de5c492eee5f admin admin] PATH_INFO: `/v3/projects/6ee68b53c65d4c4c8fc909b35095e8d6/users/4bd987b4f83140f9be828ec04809a16c/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:57.322911 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71650274-eef3-4fff-bd79-de5c492eee5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ee68b53c65d4c4c8fc909b35095e8d6, user_id=4bd987b4f83140f9be828ec04809a16c, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:57.326033 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71650274-eef3-4fff-bd79-de5c492eee5f admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:57.348096 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 912/1809] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:58:57 2022] PUT /identity/v3/projects/6ee68b53c65d4c4c8fc909b35095e8d6/users/4bd987b4f83140f9be828ec04809a16c/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 14:58:57.360287 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89bd8fc-de64-49a2-8689-46a10607eb19 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:57.361313 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89bd8fc-de64-49a2-8689-46a10607eb19 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:57.361313 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89bd8fc-de64-49a2-8689-46a10607eb19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:57.390272 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71650274-eef3-4fff-bd79-de5c492eee5f admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:57.393042 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-71650274-eef3-4fff-bd79-de5c492eee5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:57.411662 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-e89bd8fc-de64-49a2-8689-46a10607eb19 None None] MFA Rules not processed for user `4bd987b4f83140f9be828ec04809a16c`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:57.414713 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-e89bd8fc-de64-49a2-8689-46a10607eb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:57.519050 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 898/1810] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:57 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:57.531652 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89bd8fc-de64-49a2-8689-46a10607eb19 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:57.533419 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-e89bd8fc-de64-49a2-8689-46a10607eb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:57.559477 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71650274-eef3-4fff-bd79-de5c492eee5f admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:57.563798 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b342f133-0b36-43ec-90f6-89df1808cb76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:57.564461 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342f133-0b36-43ec-90f6-89df1808cb76 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:57.564780 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342f133-0b36-43ec-90f6-89df1808cb76 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:57.565078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342f133-0b36-43ec-90f6-89df1808cb76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:57.684534 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89bd8fc-de64-49a2-8689-46a10607eb19 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:57.694542 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c863e60-7544-429d-b273-5348db886a49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:57.695135 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c863e60-7544-429d-b273-5348db886a49 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:57.697563 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c863e60-7544-429d-b273-5348db886a49 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:57.697883 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c863e60-7544-429d-b273-5348db886a49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:57.721284 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c863e60-7544-429d-b273-5348db886a49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:57.726168 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c863e60-7544-429d-b273-5348db886a49 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:57.755598 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 899/1811] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:58:57 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 14:58:57.865822 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b342f133-0b36-43ec-90f6-89df1808cb76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:57.872856 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b342f133-0b36-43ec-90f6-89df1808cb76 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:57.906418 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 913/1812] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:58:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:58:57.989409 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c863e60-7544-429d-b273-5348db886a49 service cinder] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:58.003914 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c863e60-7544-429d-b273-5348db886a49 service cinder] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:58.019004 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399424cb-8dac-437f-ad67-b57c68477764 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:58.021883 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399424cb-8dac-437f-ad67-b57c68477764 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:58.023375 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399424cb-8dac-437f-ad67-b57c68477764 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:58.034489 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399424cb-8dac-437f-ad67-b57c68477764 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:58.038424 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-399424cb-8dac-437f-ad67-b57c68477764 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:58.205658 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399424cb-8dac-437f-ad67-b57c68477764 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:58.208165 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399424cb-8dac-437f-ad67-b57c68477764 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:58.248035 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 900/1813] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:58:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:58:58.685382 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b342f133-0b36-43ec-90f6-89df1808cb76 service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:58.709407 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b342f133-0b36-43ec-90f6-89df1808cb76 service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:58.735776 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2810f7cb-dad3-40fb-8e2a-3e5f8ca5504d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:58.739797 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810f7cb-dad3-40fb-8e2a-3e5f8ca5504d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:58.740079 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810f7cb-dad3-40fb-8e2a-3e5f8ca5504d admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:58.740487 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810f7cb-dad3-40fb-8e2a-3e5f8ca5504d admin admin] PATH_INFO: `/v3/users/84713df215394af981221803bc03cb7c` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:58.744356 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2810f7cb-dad3-40fb-8e2a-3e5f8ca5504d admin admin] RBAC: Authorizing `identity:delete_user(user_id=84713df215394af981221803bc03cb7c)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:58.747416 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2810f7cb-dad3-40fb-8e2a-3e5f8ca5504d admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:58.944458 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-2810f7cb-dad3-40fb-8e2a-3e5f8ca5504d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84713df215394af981221803bc03cb7c'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:58.969877 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-2810f7cb-dad3-40fb-8e2a-3e5f8ca5504d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84713df215394af981221803bc03cb7c'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:58.990294 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-2810f7cb-dad3-40fb-8e2a-3e5f8ca5504d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84713df215394af981221803bc03cb7c'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:58.991617 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-2810f7cb-dad3-40fb-8e2a-3e5f8ca5504d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84713df215394af981221803bc03cb7c'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:58.993130 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-2810f7cb-dad3-40fb-8e2a-3e5f8ca5504d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84713df215394af981221803bc03cb7c'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:59.017162 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399424cb-8dac-437f-ad67-b57c68477764 service cinder] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:59.018751 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-399424cb-8dac-437f-ad67-b57c68477764 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:59.052664 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 914/1814] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 14:58:58 2022] DELETE /identity/v3/users/84713df215394af981221803bc03cb7c => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:59.057317 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399424cb-8dac-437f-ad67-b57c68477764 service cinder] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:59.062831 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1c5fe-c953-4385-aee8-787c8bd9fe4e None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:59.063799 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1c5fe-c953-4385-aee8-787c8bd9fe4e None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:59.064175 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1c5fe-c953-4385-aee8-787c8bd9fe4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:59.135400 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-57b1c5fe-c953-4385-aee8-787c8bd9fe4e None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:58:59.139230 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-57b1c5fe-c953-4385-aee8-787c8bd9fe4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:59.195009 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195ea06e-6d0a-452f-b142-d8bc82bf6110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:59.195907 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ea06e-6d0a-452f-b142-d8bc82bf6110 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:59.196193 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ea06e-6d0a-452f-b142-d8bc82bf6110 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:59.197030 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ea06e-6d0a-452f-b142-d8bc82bf6110 admin admin] PATH_INFO: `/v3/projects/ca1c4a09ef394697bbc8317373a4c547` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:59.199607 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195ea06e-6d0a-452f-b142-d8bc82bf6110 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca1c4a09ef394697bbc8317373a4c547)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:59.202216 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195ea06e-6d0a-452f-b142-d8bc82bf6110 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:59.296247 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-195ea06e-6d0a-452f-b142-d8bc82bf6110 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca1c4a09ef394697bbc8317373a4c547', 'request_id': 'req-195ea06e-6d0a-452f-b142-d8bc82bf6110'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:59.303594 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-195ea06e-6d0a-452f-b142-d8bc82bf6110 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca1c4a09ef394697bbc8317373a4c547', 'request_id': 'req-195ea06e-6d0a-452f-b142-d8bc82bf6110'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:59.306443 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 915/1815] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:58:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:58:59.314119 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-195ea06e-6d0a-452f-b142-d8bc82bf6110 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca1c4a09ef394697bbc8317373a4c547', 'request_id': 'req-195ea06e-6d0a-452f-b142-d8bc82bf6110'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:59.329017 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-195ea06e-6d0a-452f-b142-d8bc82bf6110 admin admin] The token cache is being invalidate because project ca1c4a09ef394697bbc8317373a4c547 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 14:58:59.329381 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-195ea06e-6d0a-452f-b142-d8bc82bf6110 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:59.330653 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-195ea06e-6d0a-452f-b142-d8bc82bf6110 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:59.332183 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 901/1816] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 14:58:59 2022] DELETE /identity/v3/projects/ca1c4a09ef394697bbc8317373a4c547 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 14:58:59.336040 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b1c5fe-c953-4385-aee8-787c8bd9fe4e None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:59.338471 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-57b1c5fe-c953-4385-aee8-787c8bd9fe4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:59.352332 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b1c5fe-c953-4385-aee8-787c8bd9fe4e None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:59.389331 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195ea06e-6d0a-452f-b142-d8bc82bf6110 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:59.390894 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-195ea06e-6d0a-452f-b142-d8bc82bf6110 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:59.464778 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe97d52-9602-4c98-ba63-3774c1e9e541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:59.468925 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe97d52-9602-4c98-ba63-3774c1e9e541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:59.469145 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe97d52-9602-4c98-ba63-3774c1e9e541 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:59.469434 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe97d52-9602-4c98-ba63-3774c1e9e541 admin admin] PATH_INFO: `/v3/users/08c621d05f5748cf8b7d1eb9f03d7f1e` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:59.471047 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe97d52-9602-4c98-ba63-3774c1e9e541 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08c621d05f5748cf8b7d1eb9f03d7f1e)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:59.474926 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe97d52-9602-4c98-ba63-3774c1e9e541 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:59.533968 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195ea06e-6d0a-452f-b142-d8bc82bf6110 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:59.548849 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d94d27-cfa5-4770-af27-ce7a3506d3b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:59.549932 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d94d27-cfa5-4770-af27-ce7a3506d3b6 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:59.550410 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d94d27-cfa5-4770-af27-ce7a3506d3b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:59.550899 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d94d27-cfa5-4770-af27-ce7a3506d3b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:59.562972 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-a8d94d27-cfa5-4770-af27-ce7a3506d3b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:59.619267 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d94d27-cfa5-4770-af27-ce7a3506d3b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:59.626650 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d94d27-cfa5-4770-af27-ce7a3506d3b6 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:58:59.651993 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ebe97d52-9602-4c98-ba63-3774c1e9e541 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08c621d05f5748cf8b7d1eb9f03d7f1e'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:59.673221 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 902/1817] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:58:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 14:58:59.693793 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ebe97d52-9602-4c98-ba63-3774c1e9e541 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08c621d05f5748cf8b7d1eb9f03d7f1e'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:59.720703 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ebe97d52-9602-4c98-ba63-3774c1e9e541 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08c621d05f5748cf8b7d1eb9f03d7f1e'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:59.724346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ebe97d52-9602-4c98-ba63-3774c1e9e541 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08c621d05f5748cf8b7d1eb9f03d7f1e'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:59.726222 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ebe97d52-9602-4c98-ba63-3774c1e9e541 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08c621d05f5748cf8b7d1eb9f03d7f1e'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:58:59.786122 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 916/1818] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 14:58:59 2022] DELETE /identity/v3/users/08c621d05f5748cf8b7d1eb9f03d7f1e => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:58:59.848074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d94d27-cfa5-4770-af27-ce7a3506d3b6 service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:58:59.849539 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-a8d94d27-cfa5-4770-af27-ce7a3506d3b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:58:59.864299 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d94d27-cfa5-4770-af27-ce7a3506d3b6 service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:58:59.950244 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d26b0b-9dd8-4585-a837-bc41cf4aff3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:58:59.951191 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d26b0b-9dd8-4585-a837-bc41cf4aff3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:58:59.951479 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d26b0b-9dd8-4585-a837-bc41cf4aff3b admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:58:59.951791 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d26b0b-9dd8-4585-a837-bc41cf4aff3b admin admin] PATH_INFO: `/v3/users/0ee92f0fe95843c697acc52b0a55bc16` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:58:59.953386 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d26b0b-9dd8-4585-a837-bc41cf4aff3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ee92f0fe95843c697acc52b0a55bc16)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:58:59.956341 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d26b0b-9dd8-4585-a837-bc41cf4aff3b admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:00.021152 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-89d26b0b-9dd8-4585-a837-bc41cf4aff3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ee92f0fe95843c697acc52b0a55bc16'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:00.022636 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-89d26b0b-9dd8-4585-a837-bc41cf4aff3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ee92f0fe95843c697acc52b0a55bc16'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:00.033427 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-89d26b0b-9dd8-4585-a837-bc41cf4aff3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ee92f0fe95843c697acc52b0a55bc16'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:00.045787 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-89d26b0b-9dd8-4585-a837-bc41cf4aff3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ee92f0fe95843c697acc52b0a55bc16'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:00.046903 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-89d26b0b-9dd8-4585-a837-bc41cf4aff3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ee92f0fe95843c697acc52b0a55bc16'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:00.069100 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 903/1819] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 14:58:59 2022] DELETE /identity/v3/users/0ee92f0fe95843c697acc52b0a55bc16 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 14:59:00.074179 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a97681-0b30-46d9-a698-7e75679c7e95 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:00.074776 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a97681-0b30-46d9-a698-7e75679c7e95 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:00.075319 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a97681-0b30-46d9-a698-7e75679c7e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:00.099026 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-92a97681-0b30-46d9-a698-7e75679c7e95 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:00.100499 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-92a97681-0b30-46d9-a698-7e75679c7e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:00.118045 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df977e-5111-4a06-af13-3885eb84f8a2 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:00.119325 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df977e-5111-4a06-af13-3885eb84f8a2 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:00.119929 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df977e-5111-4a06-af13-3885eb84f8a2 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:00.122976 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 904/1820] 10.222.0.23 () {54 vars in 921 bytes} [Fri May 6 14:59:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:59:00.139067 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af46bd4-99fc-4736-a86e-620ebcb72bdb None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:00.140125 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af46bd4-99fc-4736-a86e-620ebcb72bdb None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:00.140560 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af46bd4-99fc-4736-a86e-620ebcb72bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:00.181614 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-1af46bd4-99fc-4736-a86e-620ebcb72bdb None None] MFA Rules not processed for user `fc8cf09642ec4e10a80badcf8b585ef8`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:00.185988 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-1af46bd4-99fc-4736-a86e-620ebcb72bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:00.288285 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 917/1821] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:00.328985 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a97681-0b30-46d9-a698-7e75679c7e95 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:00.330526 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-92a97681-0b30-46d9-a698-7e75679c7e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:00.400691 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 905/1822] 10.222.0.23 () {58 vars in 1018 bytes} [Fri May 6 14:59:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 14:59:00.413063 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af46bd4-99fc-4736-a86e-620ebcb72bdb None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:00.414487 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-1af46bd4-99fc-4736-a86e-620ebcb72bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:00.433277 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af46bd4-99fc-4736-a86e-620ebcb72bdb None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:00.440642 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10069c61-d6ab-47c6-bd0e-7b8dafbda4fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:00.441652 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10069c61-d6ab-47c6-bd0e-7b8dafbda4fd service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:00.441652 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10069c61-d6ab-47c6-bd0e-7b8dafbda4fd service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:00.441878 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10069c61-d6ab-47c6-bd0e-7b8dafbda4fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:00.455672 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a97681-0b30-46d9-a698-7e75679c7e95 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:00.456058 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10069c61-d6ab-47c6-bd0e-7b8dafbda4fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:00.457631 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10069c61-d6ab-47c6-bd0e-7b8dafbda4fd service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:00.467615 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700597cc-fee0-4839-a599-3e6c41d0e8f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:00.468300 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700597cc-fee0-4839-a599-3e6c41d0e8f6 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:00.468588 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700597cc-fee0-4839-a599-3e6c41d0e8f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:00.468907 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700597cc-fee0-4839-a599-3e6c41d0e8f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:00.497332 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700597cc-fee0-4839-a599-3e6c41d0e8f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:00.497689 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 906/1823] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:59:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 14:59:00.524781 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700597cc-fee0-4839-a599-3e6c41d0e8f6 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:00.553693 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 918/1824] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:59:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:59:00.967968 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10069c61-d6ab-47c6-bd0e-7b8dafbda4fd service nova] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:00.969915 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-10069c61-d6ab-47c6-bd0e-7b8dafbda4fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:01.000861 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10069c61-d6ab-47c6-bd0e-7b8dafbda4fd service nova] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:01.010858 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9e7132-5846-4514-96e5-6630c6aea6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:01.012918 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e7132-5846-4514-96e5-6630c6aea6f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:01.013312 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e7132-5846-4514-96e5-6630c6aea6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:01.013601 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e7132-5846-4514-96e5-6630c6aea6f6 admin admin] PATH_INFO: `/v3/projects/f96882caf2fb4424bf4bd786bbd56af0` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:01.014951 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9e7132-5846-4514-96e5-6630c6aea6f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f96882caf2fb4424bf4bd786bbd56af0)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:01.017322 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9e7132-5846-4514-96e5-6630c6aea6f6 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:01.091095 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-9f9e7132-5846-4514-96e5-6630c6aea6f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f96882caf2fb4424bf4bd786bbd56af0', 'request_id': 'req-9f9e7132-5846-4514-96e5-6630c6aea6f6'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:01.097140 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-9f9e7132-5846-4514-96e5-6630c6aea6f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f96882caf2fb4424bf4bd786bbd56af0', 'request_id': 'req-9f9e7132-5846-4514-96e5-6630c6aea6f6'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:01.106636 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-9f9e7132-5846-4514-96e5-6630c6aea6f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f96882caf2fb4424bf4bd786bbd56af0', 'request_id': 'req-9f9e7132-5846-4514-96e5-6630c6aea6f6'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:01.124183 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-9f9e7132-5846-4514-96e5-6630c6aea6f6 admin admin] The token cache is being invalidate because project f96882caf2fb4424bf4bd786bbd56af0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 14:59:01.124588 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-9f9e7132-5846-4514-96e5-6630c6aea6f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:01.125330 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-9f9e7132-5846-4514-96e5-6630c6aea6f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:01.127277 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 907/1825] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 14:59:00 2022] DELETE /identity/v3/projects/f96882caf2fb4424bf4bd786bbd56af0 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 14:59:01.131218 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700597cc-fee0-4839-a599-3e6c41d0e8f6 service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:01.142909 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700597cc-fee0-4839-a599-3e6c41d0e8f6 service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:01.230550 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d5d0e7-bc8f-4393-9176-5f132a7659bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:01.231456 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5d0e7-bc8f-4393-9176-5f132a7659bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:01.231791 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5d0e7-bc8f-4393-9176-5f132a7659bd admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:01.232072 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5d0e7-bc8f-4393-9176-5f132a7659bd admin admin] PATH_INFO: `/v3/users/4bd987b4f83140f9be828ec04809a16c` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:01.233373 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d5d0e7-bc8f-4393-9176-5f132a7659bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bd987b4f83140f9be828ec04809a16c)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:01.235529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d5d0e7-bc8f-4393-9176-5f132a7659bd admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:01.299744 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-46d5d0e7-bc8f-4393-9176-5f132a7659bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bd987b4f83140f9be828ec04809a16c'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:01.314867 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-46d5d0e7-bc8f-4393-9176-5f132a7659bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bd987b4f83140f9be828ec04809a16c'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:01.325157 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-46d5d0e7-bc8f-4393-9176-5f132a7659bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bd987b4f83140f9be828ec04809a16c'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:01.326738 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-46d5d0e7-bc8f-4393-9176-5f132a7659bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bd987b4f83140f9be828ec04809a16c'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:01.327416 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-46d5d0e7-bc8f-4393-9176-5f132a7659bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bd987b4f83140f9be828ec04809a16c'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:01.347265 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 919/1826] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 14:59:01 2022] DELETE /identity/v3/users/4bd987b4f83140f9be828ec04809a16c => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:01.595809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9e7132-5846-4514-96e5-6630c6aea6f6 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:01.595809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-9f9e7132-5846-4514-96e5-6630c6aea6f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:01.611529 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9e7132-5846-4514-96e5-6630c6aea6f6 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:01.701045 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e553b2f4-addd-4ca2-b303-7c559618f35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:01.702604 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e553b2f4-addd-4ca2-b303-7c559618f35f admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:01.702903 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e553b2f4-addd-4ca2-b303-7c559618f35f admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:01.703184 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e553b2f4-addd-4ca2-b303-7c559618f35f admin admin] PATH_INFO: `/v3/projects/30a08c313b4745bd9e2e9a5dfe19672b` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:01.704461 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e553b2f4-addd-4ca2-b303-7c559618f35f admin admin] RBAC: Authorizing `identity:delete_project(project_id=30a08c313b4745bd9e2e9a5dfe19672b)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:01.706436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e553b2f4-addd-4ca2-b303-7c559618f35f admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:01.829776 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e553b2f4-addd-4ca2-b303-7c559618f35f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30a08c313b4745bd9e2e9a5dfe19672b', 'request_id': 'req-e553b2f4-addd-4ca2-b303-7c559618f35f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:01.842867 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e553b2f4-addd-4ca2-b303-7c559618f35f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30a08c313b4745bd9e2e9a5dfe19672b', 'request_id': 'req-e553b2f4-addd-4ca2-b303-7c559618f35f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:01.851396 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e553b2f4-addd-4ca2-b303-7c559618f35f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30a08c313b4745bd9e2e9a5dfe19672b', 'request_id': 'req-e553b2f4-addd-4ca2-b303-7c559618f35f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:01.867268 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e553b2f4-addd-4ca2-b303-7c559618f35f admin admin] The token cache is being invalidate because project 30a08c313b4745bd9e2e9a5dfe19672b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 14:59:01.867637 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e553b2f4-addd-4ca2-b303-7c559618f35f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:01.868924 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e553b2f4-addd-4ca2-b303-7c559618f35f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:01.870815 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 908/1827] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 14:59:01 2022] DELETE /identity/v3/projects/30a08c313b4745bd9e2e9a5dfe19672b => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:01.875790 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c70c1a-8f5c-4fde-8c64-556fc248e223 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:01.876504 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c70c1a-8f5c-4fde-8c64-556fc248e223 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:01.876945 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c70c1a-8f5c-4fde-8c64-556fc248e223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:01.901202 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-f6c70c1a-8f5c-4fde-8c64-556fc248e223 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:01.902446 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f6c70c1a-8f5c-4fde-8c64-556fc248e223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:01.995746 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 920/1828] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:02.005029 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc6c1e-6f53-4123-baa8-8f8835e1235c None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:02.013081 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc6c1e-6f53-4123-baa8-8f8835e1235c None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:02.013511 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc6c1e-6f53-4123-baa8-8f8835e1235c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:02.068634 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-20fc6c1e-6f53-4123-baa8-8f8835e1235c None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:02.069899 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-20fc6c1e-6f53-4123-baa8-8f8835e1235c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:02.097119 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 909/1829] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:02.097747 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c70c1a-8f5c-4fde-8c64-556fc248e223 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:02.114132 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c70c1a-8f5c-4fde-8c64-556fc248e223 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:02.121449 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f6a0dd-a65a-4e48-be6b-13b6f375ece3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:02.122380 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6a0dd-a65a-4e48-be6b-13b6f375ece3 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:02.122762 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6a0dd-a65a-4e48-be6b-13b6f375ece3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:02.123139 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6a0dd-a65a-4e48-be6b-13b6f375ece3 admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:02.124194 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f6a0dd-a65a-4e48-be6b-13b6f375ece3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:59:02.124395 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f6a0dd-a65a-4e48-be6b-13b6f375ece3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:02.131290 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f6a0dd-a65a-4e48-be6b-13b6f375ece3 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:02.159530 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 921/1830] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 14:59:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:59:02.163866 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc6c1e-6f53-4123-baa8-8f8835e1235c None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:02.180430 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc6c1e-6f53-4123-baa8-8f8835e1235c None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:02.196785 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40e9ce-05df-4489-b182-ea101135fbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:02.197904 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40e9ce-05df-4489-b182-ea101135fbce admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:02.198421 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40e9ce-05df-4489-b182-ea101135fbce admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:02.198905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40e9ce-05df-4489-b182-ea101135fbce admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:02.200568 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e40e9ce-05df-4489-b182-ea101135fbce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:02.208733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e40e9ce-05df-4489-b182-ea101135fbce admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:02.261289 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 910/1831] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:59:02 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:59:02.265133 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f6a0dd-a65a-4e48-be6b-13b6f375ece3 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:02.287023 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f6a0dd-a65a-4e48-be6b-13b6f375ece3 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:02.385619 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aafd72-f52d-4a86-b439-bdb896b8cb8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:02.386675 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aafd72-f52d-4a86-b439-bdb896b8cb8b admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:02.387103 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aafd72-f52d-4a86-b439-bdb896b8cb8b admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:02.387535 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aafd72-f52d-4a86-b439-bdb896b8cb8b admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:02.388397 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aafd72-f52d-4a86-b439-bdb896b8cb8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:02.390785 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aafd72-f52d-4a86-b439-bdb896b8cb8b admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:02.459327 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 922/1832] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:59:02 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:59:02.470186 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40e9ce-05df-4489-b182-ea101135fbce admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:02.489375 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40e9ce-05df-4489-b182-ea101135fbce admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:02.507682 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a97637-cf4a-4f12-8f11-c5f63f51b3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:02.508368 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a97637-cf4a-4f12-8f11-c5f63f51b3dd admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:02.509843 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a97637-cf4a-4f12-8f11-c5f63f51b3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:02.510410 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a97637-cf4a-4f12-8f11-c5f63f51b3dd admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:02.511375 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a97637-cf4a-4f12-8f11-c5f63f51b3dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:02.513629 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a97637-cf4a-4f12-8f11-c5f63f51b3dd admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:02.531777 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 911/1833] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:02 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:59:02.536190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aafd72-f52d-4a86-b439-bdb896b8cb8b admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:02.553275 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aafd72-f52d-4a86-b439-bdb896b8cb8b admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:02.576749 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a97637-cf4a-4f12-8f11-c5f63f51b3dd admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:02.577070 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f773e8-d1e4-43db-b058-3db11473f5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:02.577763 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f773e8-d1e4-43db-b058-3db11473f5ad admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:02.578042 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f773e8-d1e4-43db-b058-3db11473f5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:02.578506 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f773e8-d1e4-43db-b058-3db11473f5ad admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:02.578613 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-f2a97637-cf4a-4f12-8f11-c5f63f51b3dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:02.579239 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f773e8-d1e4-43db-b058-3db11473f5ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:02.581396 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f773e8-d1e4-43db-b058-3db11473f5ad admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:02.597536 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 923/1834] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:02 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:59:02.599956 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f773e8-d1e4-43db-b058-3db11473f5ad admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:02.617117 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a97637-cf4a-4f12-8f11-c5f63f51b3dd admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:02.617907 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f773e8-d1e4-43db-b058-3db11473f5ad admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:02.628528 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db4a35-80e1-4c9b-b567-d438f3e5df72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:02.632370 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db4a35-80e1-4c9b-b567-d438f3e5df72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:02.632729 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813db05a-db8e-4c86-a2a2-377839a7bb84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:02.634272 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db4a35-80e1-4c9b-b567-d438f3e5df72 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:02.634683 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813db05a-db8e-4c86-a2a2-377839a7bb84 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:02.634923 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813db05a-db8e-4c86-a2a2-377839a7bb84 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:02.635354 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813db05a-db8e-4c86-a2a2-377839a7bb84 admin admin] PATH_INFO: `/v3/projects/7556b262575047aca552e72d6828468d/users/590985dde4fa4defb550529858e16ce2/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:02.636136 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db4a35-80e1-4c9b-b567-d438f3e5df72 admin admin] PATH_INFO: `/v3/projects/d4f5a9c30c0d4ff18e3aef3314390f14` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:02.645690 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35db4a35-80e1-4c9b-b567-d438f3e5df72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4f5a9c30c0d4ff18e3aef3314390f14)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:02.647999 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35db4a35-80e1-4c9b-b567-d438f3e5df72 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:02.666013 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813db05a-db8e-4c86-a2a2-377839a7bb84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7556b262575047aca552e72d6828468d, user_id=590985dde4fa4defb550529858e16ce2, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:02.669998 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813db05a-db8e-4c86-a2a2-377839a7bb84 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:02.700666 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 924/1835] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:59:02 2022] PUT /identity/v3/projects/7556b262575047aca552e72d6828468d/users/590985dde4fa4defb550529858e16ce2/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 14:59:02.704594 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb2dc3-b635-4353-9eb5-4bd4e027886f None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:02.707392 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb2dc3-b635-4353-9eb5-4bd4e027886f None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:02.707883 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb2dc3-b635-4353-9eb5-4bd4e027886f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:02.742563 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-3ebb2dc3-b635-4353-9eb5-4bd4e027886f None None] MFA Rules not processed for user `590985dde4fa4defb550529858e16ce2`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:02.744249 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-3ebb2dc3-b635-4353-9eb5-4bd4e027886f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:02.760003 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-35db4a35-80e1-4c9b-b567-d438f3e5df72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4f5a9c30c0d4ff18e3aef3314390f14', 'request_id': 'req-35db4a35-80e1-4c9b-b567-d438f3e5df72'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:02.772500 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-35db4a35-80e1-4c9b-b567-d438f3e5df72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4f5a9c30c0d4ff18e3aef3314390f14', 'request_id': 'req-35db4a35-80e1-4c9b-b567-d438f3e5df72'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:02.791643 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-35db4a35-80e1-4c9b-b567-d438f3e5df72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4f5a9c30c0d4ff18e3aef3314390f14', 'request_id': 'req-35db4a35-80e1-4c9b-b567-d438f3e5df72'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:02.814743 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-35db4a35-80e1-4c9b-b567-d438f3e5df72 admin admin] The token cache is being invalidate because project d4f5a9c30c0d4ff18e3aef3314390f14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 14:59:02.815127 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-35db4a35-80e1-4c9b-b567-d438f3e5df72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:02.816098 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-35db4a35-80e1-4c9b-b567-d438f3e5df72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:02.822773 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 912/1836] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 14:59:02 2022] DELETE /identity/v3/projects/d4f5a9c30c0d4ff18e3aef3314390f14 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 14:59:02.826700 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db4a35-80e1-4c9b-b567-d438f3e5df72 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:02.828215 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-35db4a35-80e1-4c9b-b567-d438f3e5df72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:02.879603 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 925/1837] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:02.881519 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb2dc3-b635-4353-9eb5-4bd4e027886f None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:02.883280 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-3ebb2dc3-b635-4353-9eb5-4bd4e027886f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:02.937205 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db4a35-80e1-4c9b-b567-d438f3e5df72 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:02.949550 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db182bf6-e378-457d-a1f3-a4861dc2bf99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:02.950837 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db182bf6-e378-457d-a1f3-a4861dc2bf99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:02.951393 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db182bf6-e378-457d-a1f3-a4861dc2bf99 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:02.954082 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db182bf6-e378-457d-a1f3-a4861dc2bf99 admin admin] PATH_INFO: `/v3/users/327b46b84403423d8a01218c9700c9d2` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:02.956133 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db182bf6-e378-457d-a1f3-a4861dc2bf99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=327b46b84403423d8a01218c9700c9d2)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:02.959755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db182bf6-e378-457d-a1f3-a4861dc2bf99 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:02.984061 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb2dc3-b635-4353-9eb5-4bd4e027886f None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:02.991053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3031eb8-3546-479e-8b0c-148e6ca4ca27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:02.991885 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3031eb8-3546-479e-8b0c-148e6ca4ca27 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:02.992275 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3031eb8-3546-479e-8b0c-148e6ca4ca27 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:02.992683 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3031eb8-3546-479e-8b0c-148e6ca4ca27 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:02.994136 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3031eb8-3546-479e-8b0c-148e6ca4ca27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:02.996367 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3031eb8-3546-479e-8b0c-148e6ca4ca27 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:03.051271 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 926/1838] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:59:02 2022] POST /identity/v3/projects => generated 387 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:59:03.056004 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3031eb8-3546-479e-8b0c-148e6ca4ca27 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:03.125856 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-db182bf6-e378-457d-a1f3-a4861dc2bf99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '327b46b84403423d8a01218c9700c9d2'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:03.131626 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-db182bf6-e378-457d-a1f3-a4861dc2bf99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '327b46b84403423d8a01218c9700c9d2'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:03.158960 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-db182bf6-e378-457d-a1f3-a4861dc2bf99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '327b46b84403423d8a01218c9700c9d2'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:03.171643 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-db182bf6-e378-457d-a1f3-a4861dc2bf99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '327b46b84403423d8a01218c9700c9d2'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:03.173883 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-db182bf6-e378-457d-a1f3-a4861dc2bf99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '327b46b84403423d8a01218c9700c9d2'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:03.177890 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3031eb8-3546-479e-8b0c-148e6ca4ca27 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:03.186453 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb17f6b-6f67-4571-9fa6-dafa5f396c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:03.187634 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb17f6b-6f67-4571-9fa6-dafa5f396c1b admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:03.188126 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb17f6b-6f67-4571-9fa6-dafa5f396c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:03.188515 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb17f6b-6f67-4571-9fa6-dafa5f396c1b admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:03.189784 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb17f6b-6f67-4571-9fa6-dafa5f396c1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:03.192594 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb17f6b-6f67-4571-9fa6-dafa5f396c1b admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:03.200733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 913/1839] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 14:59:02 2022] DELETE /identity/v3/users/327b46b84403423d8a01218c9700c9d2 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:03.220299 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db182bf6-e378-457d-a1f3-a4861dc2bf99 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:03.223037 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 927/1840] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:59:03 2022] POST /identity/v3/users => generated 294 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 14:59:03.225623 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-db182bf6-e378-457d-a1f3-a4861dc2bf99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:03.226885 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb17f6b-6f67-4571-9fa6-dafa5f396c1b admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:03.228939 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-0eb17f6b-6f67-4571-9fa6-dafa5f396c1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:03.242672 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb17f6b-6f67-4571-9fa6-dafa5f396c1b admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:03.321333 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db182bf6-e378-457d-a1f3-a4861dc2bf99 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:03.330147 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234df12-f06e-40b9-a9bb-3f6af40536cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:03.331394 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234df12-f06e-40b9-a9bb-3f6af40536cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:03.331678 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234df12-f06e-40b9-a9bb-3f6af40536cf admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:03.332153 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234df12-f06e-40b9-a9bb-3f6af40536cf admin admin] PATH_INFO: `/v3/projects/6ee68b53c65d4c4c8fc909b35095e8d6` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:03.333838 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8234df12-f06e-40b9-a9bb-3f6af40536cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ee68b53c65d4c4c8fc909b35095e8d6)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:03.337645 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8234df12-f06e-40b9-a9bb-3f6af40536cf admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:03.377257 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c4711-af14-4371-ab8e-a9e29df290f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:03.379282 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c4711-af14-4371-ab8e-a9e29df290f4 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:03.381217 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c4711-af14-4371-ab8e-a9e29df290f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:03.381726 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c4711-af14-4371-ab8e-a9e29df290f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:03.384239 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c4711-af14-4371-ab8e-a9e29df290f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:03.387869 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c4711-af14-4371-ab8e-a9e29df290f4 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:03.416943 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 928/1841] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:03 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:59:03.420753 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c4711-af14-4371-ab8e-a9e29df290f4 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:03.421146 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8234df12-f06e-40b9-a9bb-3f6af40536cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ee68b53c65d4c4c8fc909b35095e8d6', 'request_id': 'req-8234df12-f06e-40b9-a9bb-3f6af40536cf'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:03.427399 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8234df12-f06e-40b9-a9bb-3f6af40536cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ee68b53c65d4c4c8fc909b35095e8d6', 'request_id': 'req-8234df12-f06e-40b9-a9bb-3f6af40536cf'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:03.435601 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c4711-af14-4371-ab8e-a9e29df290f4 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:03.440486 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8234df12-f06e-40b9-a9bb-3f6af40536cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ee68b53c65d4c4c8fc909b35095e8d6', 'request_id': 'req-8234df12-f06e-40b9-a9bb-3f6af40536cf'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:03.461513 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8234df12-f06e-40b9-a9bb-3f6af40536cf admin admin] The token cache is being invalidate because project 6ee68b53c65d4c4c8fc909b35095e8d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 14:59:03.464456 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8234df12-f06e-40b9-a9bb-3f6af40536cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:03.465472 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8234df12-f06e-40b9-a9bb-3f6af40536cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:03.469043 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 914/1842] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 14:59:03 2022] DELETE /identity/v3/projects/6ee68b53c65d4c4c8fc909b35095e8d6 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:03.475656 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1125e4e-8668-4a77-a13a-df4927b2ac2e None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:03.476194 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1125e4e-8668-4a77-a13a-df4927b2ac2e None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:03.476672 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1125e4e-8668-4a77-a13a-df4927b2ac2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:03.527857 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-d1125e4e-8668-4a77-a13a-df4927b2ac2e None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:03.529119 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d1125e4e-8668-4a77-a13a-df4927b2ac2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:03.578337 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df87fe94-1e22-4238-98da-f09df9dca872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:03.579178 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87fe94-1e22-4238-98da-f09df9dca872 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:03.579585 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87fe94-1e22-4238-98da-f09df9dca872 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:03.581558 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87fe94-1e22-4238-98da-f09df9dca872 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:03.582581 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df87fe94-1e22-4238-98da-f09df9dca872 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:03.585218 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df87fe94-1e22-4238-98da-f09df9dca872 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:03.606195 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 929/1843] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:03 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 06 14:59:03.607359 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df87fe94-1e22-4238-98da-f09df9dca872 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:03.609623 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-df87fe94-1e22-4238-98da-f09df9dca872 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:03.622335 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 915/1844] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:03.628796 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904ee11-1227-48f7-ab5f-a677568a3821 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:03.629191 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904ee11-1227-48f7-ab5f-a677568a3821 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:03.629638 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904ee11-1227-48f7-ab5f-a677568a3821 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:03.631699 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 916/1845] 10.222.0.23 () {54 vars in 921 bytes} [Fri May 6 14:59:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 14:59:03.633061 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf81ecb-2326-4a9c-95e1-2f55eff073f5 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:03.633471 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf81ecb-2326-4a9c-95e1-2f55eff073f5 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:03.634135 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf81ecb-2326-4a9c-95e1-2f55eff073f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:03.676086 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-0cf81ecb-2326-4a9c-95e1-2f55eff073f5 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:03.677508 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-0cf81ecb-2326-4a9c-95e1-2f55eff073f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:03.687537 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 917/1846] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:03.689061 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0336bf17-a0af-4c4a-b6d0-776f0a724c96 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:03.689449 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0336bf17-a0af-4c4a-b6d0-776f0a724c96 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:03.689824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0336bf17-a0af-4c4a-b6d0-776f0a724c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:03.690737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df87fe94-1e22-4238-98da-f09df9dca872 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:03.717097 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-0336bf17-a0af-4c4a-b6d0-776f0a724c96 None None] MFA Rules not processed for user `fc8cf09642ec4e10a80badcf8b585ef8`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:03.721074 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac55b1f8-4adb-4d70-9045-cea029961403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:03.721994 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac55b1f8-4adb-4d70-9045-cea029961403 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:03.722284 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac55b1f8-4adb-4d70-9045-cea029961403 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:03.723105 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac55b1f8-4adb-4d70-9045-cea029961403 admin admin] PATH_INFO: `/v3/projects/5011f1a7114f49678c6da0756ce0a3f0/users/d78e95c546ae4f8584bcacace9b2ce8c/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:03.730746 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-0336bf17-a0af-4c4a-b6d0-776f0a724c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:03.755907 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac55b1f8-4adb-4d70-9045-cea029961403 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5011f1a7114f49678c6da0756ce0a3f0, user_id=d78e95c546ae4f8584bcacace9b2ce8c, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:03.767106 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac55b1f8-4adb-4d70-9045-cea029961403 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:03.796533 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 930/1847] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:59:03 2022] PUT /identity/v3/projects/5011f1a7114f49678c6da0756ce0a3f0/users/d78e95c546ae4f8584bcacace9b2ce8c/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:03.797702 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac55b1f8-4adb-4d70-9045-cea029961403 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:03.856752 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 918/1848] 10.222.0.23 () {58 vars in 1018 bytes} [Fri May 6 14:59:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:03.858752 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e89b7b-3f14-4cd5-80b2-7707564b866b None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:03.859107 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e89b7b-3f14-4cd5-80b2-7707564b866b None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:03.859783 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e89b7b-3f14-4cd5-80b2-7707564b866b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:03.942148 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-05e89b7b-3f14-4cd5-80b2-7707564b866b None None] MFA Rules not processed for user `d78e95c546ae4f8584bcacace9b2ce8c`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:03.943382 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-05e89b7b-3f14-4cd5-80b2-7707564b866b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:03.985605 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac55b1f8-4adb-4d70-9045-cea029961403 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:03.998730 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b79e5-1600-434d-8c52-4c3da6ea2445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:03.999442 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b79e5-1600-434d-8c52-4c3da6ea2445 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:03.999743 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b79e5-1600-434d-8c52-4c3da6ea2445 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:04.001660 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b79e5-1600-434d-8c52-4c3da6ea2445 admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:04.001857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40b79e5-1600-434d-8c52-4c3da6ea2445 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:59:04.002790 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40b79e5-1600-434d-8c52-4c3da6ea2445 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:04.005902 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40b79e5-1600-434d-8c52-4c3da6ea2445 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:04.037702 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 931/1849] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 14:59:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:59:04.041447 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b79e5-1600-434d-8c52-4c3da6ea2445 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:04.044895 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f40b79e5-1600-434d-8c52-4c3da6ea2445 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:04.071732 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b79e5-1600-434d-8c52-4c3da6ea2445 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:04.078349 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b5198-eb15-46c4-a748-bf856d267568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:04.079750 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b5198-eb15-46c4-a748-bf856d267568 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:04.079750 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b5198-eb15-46c4-a748-bf856d267568 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:04.080053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b5198-eb15-46c4-a748-bf856d267568 admin admin] PATH_INFO: `/v3/projects/952f23964070490ca0912354e4a1ef64` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:04.081275 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680b5198-eb15-46c4-a748-bf856d267568 admin admin] RBAC: Authorizing `identity:delete_project(project_id=952f23964070490ca0912354e4a1ef64)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:04.083901 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680b5198-eb15-46c4-a748-bf856d267568 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:04.107344 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 919/1850] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:04.109668 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e89b7b-3f14-4cd5-80b2-7707564b866b None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:04.113595 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-05e89b7b-3f14-4cd5-80b2-7707564b866b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:04.153960 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-680b5198-eb15-46c4-a748-bf856d267568 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '952f23964070490ca0912354e4a1ef64', 'request_id': 'req-680b5198-eb15-46c4-a748-bf856d267568'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:04.160792 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-680b5198-eb15-46c4-a748-bf856d267568 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '952f23964070490ca0912354e4a1ef64', 'request_id': 'req-680b5198-eb15-46c4-a748-bf856d267568'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:04.170226 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-680b5198-eb15-46c4-a748-bf856d267568 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '952f23964070490ca0912354e4a1ef64', 'request_id': 'req-680b5198-eb15-46c4-a748-bf856d267568'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:04.185116 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-680b5198-eb15-46c4-a748-bf856d267568 admin admin] The token cache is being invalidate because project 952f23964070490ca0912354e4a1ef64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 14:59:04.185772 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-680b5198-eb15-46c4-a748-bf856d267568 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:04.186302 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-680b5198-eb15-46c4-a748-bf856d267568 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:04.191710 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 932/1851] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 14:59:04 2022] DELETE /identity/v3/projects/952f23964070490ca0912354e4a1ef64 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:04.191710 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b5198-eb15-46c4-a748-bf856d267568 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:04.192435 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-680b5198-eb15-46c4-a748-bf856d267568 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:04.210374 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e89b7b-3f14-4cd5-80b2-7707564b866b None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:04.284953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b5198-eb15-46c4-a748-bf856d267568 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:04.290326 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd446ab-d5ca-4442-9c3d-ff1b3fb9f48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:04.291498 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd446ab-d5ca-4442-9c3d-ff1b3fb9f48c admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:04.291801 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd446ab-d5ca-4442-9c3d-ff1b3fb9f48c admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:04.292083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd446ab-d5ca-4442-9c3d-ff1b3fb9f48c admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:04.292825 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd446ab-d5ca-4442-9c3d-ff1b3fb9f48c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:04.294858 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd446ab-d5ca-4442-9c3d-ff1b3fb9f48c admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:04.300260 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9afb3e-2f9e-4d62-a433-292b5345c11a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:04.301230 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9afb3e-2f9e-4d62-a433-292b5345c11a service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:04.301651 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9afb3e-2f9e-4d62-a433-292b5345c11a service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:04.301934 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9afb3e-2f9e-4d62-a433-292b5345c11a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:04.303998 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-0f9afb3e-2f9e-4d62-a433-292b5345c11a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:04.318040 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9afb3e-2f9e-4d62-a433-292b5345c11a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:04.320081 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9afb3e-2f9e-4d62-a433-292b5345c11a service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:04.328171 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 933/1852] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:59:04 2022] POST /identity/v3/projects => generated 383 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:59:04.329279 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd446ab-d5ca-4442-9c3d-ff1b3fb9f48c admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:04.332588 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-cbd446ab-d5ca-4442-9c3d-ff1b3fb9f48c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:04.340275 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 920/1853] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:59:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:59:04.343189 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082cbc8e-fc53-45df-abb2-cb4e6298270f None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:04.343505 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082cbc8e-fc53-45df-abb2-cb4e6298270f None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:04.343802 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082cbc8e-fc53-45df-abb2-cb4e6298270f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:04.378285 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-082cbc8e-fc53-45df-abb2-cb4e6298270f None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:04.379516 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-082cbc8e-fc53-45df-abb2-cb4e6298270f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:04.443561 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd446ab-d5ca-4442-9c3d-ff1b3fb9f48c admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:04.449273 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c286ba-ef26-49b6-8353-35265388e832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:04.450055 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c286ba-ef26-49b6-8353-35265388e832 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:04.450348 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c286ba-ef26-49b6-8353-35265388e832 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:04.450628 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c286ba-ef26-49b6-8353-35265388e832 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:04.451321 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c286ba-ef26-49b6-8353-35265388e832 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:04.453268 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c286ba-ef26-49b6-8353-35265388e832 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:04.466733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 921/1854] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:04.466733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082cbc8e-fc53-45df-abb2-cb4e6298270f None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:04.476203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082cbc8e-fc53-45df-abb2-cb4e6298270f None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:04.481643 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683def49-4392-48ac-a8de-c97e8bad01b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:04.482312 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683def49-4392-48ac-a8de-c97e8bad01b8 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:04.482601 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683def49-4392-48ac-a8de-c97e8bad01b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:04.482879 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683def49-4392-48ac-a8de-c97e8bad01b8 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:04.483561 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683def49-4392-48ac-a8de-c97e8bad01b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:04.485538 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683def49-4392-48ac-a8de-c97e8bad01b8 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:04.489931 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 934/1855] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:59:04 2022] POST /identity/v3/projects => generated 387 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:59:04.489931 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4c8df-797a-44e2-9399-d3922558c4db None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:04.489931 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4c8df-797a-44e2-9399-d3922558c4db None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:04.489931 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4c8df-797a-44e2-9399-d3922558c4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:04.511047 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 922/1856] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:59:04 2022] POST /identity/v3/users => generated 292 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:59:04.511238 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-8aa4c8df-797a-44e2-9399-d3922558c4db None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:04.511355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683def49-4392-48ac-a8de-c97e8bad01b8 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:04.511444 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-8aa4c8df-797a-44e2-9399-d3922558c4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:04.518680 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683def49-4392-48ac-a8de-c97e8bad01b8 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:04.571753 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 935/1857] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:04.572638 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa4c8df-797a-44e2-9399-d3922558c4db None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:04.577655 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db3845-3fd6-4c60-944a-2a6ff9bf2228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:04.578325 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db3845-3fd6-4c60-944a-2a6ff9bf2228 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:04.578609 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db3845-3fd6-4c60-944a-2a6ff9bf2228 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:04.578891 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db3845-3fd6-4c60-944a-2a6ff9bf2228 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:04.579578 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35db3845-3fd6-4c60-944a-2a6ff9bf2228 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:04.581530 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35db3845-3fd6-4c60-944a-2a6ff9bf2228 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:04.582014 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa4c8df-797a-44e2-9399-d3922558c4db None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:04.586696 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1167802-b131-41de-9fe0-40daf776555d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:04.587870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1167802-b131-41de-9fe0-40daf776555d admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:04.588153 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1167802-b131-41de-9fe0-40daf776555d admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:04.588432 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1167802-b131-41de-9fe0-40daf776555d admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:04.589183 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1167802-b131-41de-9fe0-40daf776555d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:04.590765 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1167802-b131-41de-9fe0-40daf776555d admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:04.604991 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 936/1858] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:59:04.605172 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 923/1859] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:59:04 2022] POST /identity/v3/users => generated 300 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:59:04.605265 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1167802-b131-41de-9fe0-40daf776555d admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:04.607900 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db3845-3fd6-4c60-944a-2a6ff9bf2228 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:04.613837 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1167802-b131-41de-9fe0-40daf776555d admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:04.618715 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db3845-3fd6-4c60-944a-2a6ff9bf2228 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:04.620024 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9120406a-56bb-4859-b546-b9539f2aa5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:04.620884 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120406a-56bb-4859-b546-b9539f2aa5fb admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:04.621270 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120406a-56bb-4859-b546-b9539f2aa5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:04.621482 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120406a-56bb-4859-b546-b9539f2aa5fb admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:04.622396 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9120406a-56bb-4859-b546-b9539f2aa5fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:59:04.622773 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9120406a-56bb-4859-b546-b9539f2aa5fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:04.624190 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e275973f-a56b-4c61-b612-3c55cdb71267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:04.624434 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9120406a-56bb-4859-b546-b9539f2aa5fb admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:04.624906 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e275973f-a56b-4c61-b612-3c55cdb71267 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:04.625195 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e275973f-a56b-4c61-b612-3c55cdb71267 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:04.625475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e275973f-a56b-4c61-b612-3c55cdb71267 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:04.626200 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e275973f-a56b-4c61-b612-3c55cdb71267 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:04.627785 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e275973f-a56b-4c61-b612-3c55cdb71267 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:04.637708 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 937/1860] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 14:59:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:59:04.638423 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 924/1861] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:59:04.638648 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9120406a-56bb-4859-b546-b9539f2aa5fb admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:04.641952 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e275973f-a56b-4c61-b612-3c55cdb71267 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:04.648925 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9120406a-56bb-4859-b546-b9539f2aa5fb admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:04.652136 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e275973f-a56b-4c61-b612-3c55cdb71267 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:04.655066 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f308b484-1160-465a-9b31-dd69cf543fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:04.655984 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308b484-1160-465a-9b31-dd69cf543fd8 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:04.656264 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308b484-1160-465a-9b31-dd69cf543fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:04.656533 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308b484-1160-465a-9b31-dd69cf543fd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:04.657110 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a30cf-8fa0-4197-b294-cbce4fa73d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:04.657332 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f308b484-1160-465a-9b31-dd69cf543fd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:04.657947 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a30cf-8fa0-4197-b294-cbce4fa73d26 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:04.658068 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a30cf-8fa0-4197-b294-cbce4fa73d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:04.658315 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a30cf-8fa0-4197-b294-cbce4fa73d26 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:04.658878 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f308b484-1160-465a-9b31-dd69cf543fd8 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:04.659045 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92a30cf-8fa0-4197-b294-cbce4fa73d26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:04.660935 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92a30cf-8fa0-4197-b294-cbce4fa73d26 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:04.670351 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 938/1862] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:59:04.671416 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f308b484-1160-465a-9b31-dd69cf543fd8 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:04.682190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f308b484-1160-465a-9b31-dd69cf543fd8 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:04.687867 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25d2096-1837-427e-ad51-6cd730b96371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:04.688808 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d2096-1837-427e-ad51-6cd730b96371 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:04.689092 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d2096-1837-427e-ad51-6cd730b96371 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:04.689363 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d2096-1837-427e-ad51-6cd730b96371 admin admin] PATH_INFO: `/v3/projects/72ed39139a59446fa07d5e8812ee0d41/users/1d6b2f1beece46869291ed5d4fa5f6e9/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:04.690030 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 925/1863] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:59:04 2022] POST /identity/v3/projects => generated 361 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 14:59:04.691070 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a30cf-8fa0-4197-b294-cbce4fa73d26 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:04.708202 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25d2096-1837-427e-ad51-6cd730b96371 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72ed39139a59446fa07d5e8812ee0d41, user_id=1d6b2f1beece46869291ed5d4fa5f6e9, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:04.710943 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25d2096-1837-427e-ad51-6cd730b96371 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:04.730311 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 939/1864] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:59:04 2022] PUT /identity/v3/projects/72ed39139a59446fa07d5e8812ee0d41/users/1d6b2f1beece46869291ed5d4fa5f6e9/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:04.733602 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25d2096-1837-427e-ad51-6cd730b96371 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:04.764463 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a30cf-8fa0-4197-b294-cbce4fa73d26 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:04.797729 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25d2096-1837-427e-ad51-6cd730b96371 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:04.802380 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe7fed8-5aa8-40a0-a1f4-08dea84cbd2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:04.803056 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7fed8-5aa8-40a0-a1f4-08dea84cbd2b admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:04.803339 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7fed8-5aa8-40a0-a1f4-08dea84cbd2b admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:04.803617 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7fed8-5aa8-40a0-a1f4-08dea84cbd2b admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:04.804333 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe7fed8-5aa8-40a0-a1f4-08dea84cbd2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:04.806831 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe7fed8-5aa8-40a0-a1f4-08dea84cbd2b admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:04.823682 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d003a091-7fc5-43c5-a355-55b6e467351b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:04.824684 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003a091-7fc5-43c5-a355-55b6e467351b admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:04.824999 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003a091-7fc5-43c5-a355-55b6e467351b admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:04.825277 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003a091-7fc5-43c5-a355-55b6e467351b admin admin] PATH_INFO: `/v3/projects/b4e66868534f4df09eaa1e975f2890e1/users/a6932dd687284d78a5792c7b1d0ade8b/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:04.830657 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 940/1865] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:59:04 2022] POST /identity/v3/users => generated 281 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:59:04.830657 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b26fe-79bd-4656-b85d-2017bac365d1 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:04.830657 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b26fe-79bd-4656-b85d-2017bac365d1 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:04.830657 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b26fe-79bd-4656-b85d-2017bac365d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:04.841319 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d003a091-7fc5-43c5-a355-55b6e467351b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4e66868534f4df09eaa1e975f2890e1, user_id=a6932dd687284d78a5792c7b1d0ade8b, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:04.843951 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d003a091-7fc5-43c5-a355-55b6e467351b admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:04.855975 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-187b26fe-79bd-4656-b85d-2017bac365d1 None None] MFA Rules not processed for user `1d6b2f1beece46869291ed5d4fa5f6e9`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:04.857203 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-187b26fe-79bd-4656-b85d-2017bac365d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:04.858444 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 926/1866] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:59:04 2022] PUT /identity/v3/projects/b4e66868534f4df09eaa1e975f2890e1/users/a6932dd687284d78a5792c7b1d0ade8b/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:04.859251 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d003a091-7fc5-43c5-a355-55b6e467351b admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:04.929749 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 941/1867] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:04.931361 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b26fe-79bd-4656-b85d-2017bac365d1 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:04.936859 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d003a091-7fc5-43c5-a355-55b6e467351b admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:04.944145 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b26fe-79bd-4656-b85d-2017bac365d1 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:04.944307 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d75b0f-a545-471c-8a0f-a8ee2b61c0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:04.945175 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d75b0f-a545-471c-8a0f-a8ee2b61c0f0 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:04.945391 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d75b0f-a545-471c-8a0f-a8ee2b61c0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:04.945768 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d75b0f-a545-471c-8a0f-a8ee2b61c0f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:04.947280 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d75b0f-a545-471c-8a0f-a8ee2b61c0f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:04.949132 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d75b0f-a545-471c-8a0f-a8ee2b61c0f0 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:04.949739 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406167c5-6dfc-4606-9322-6e9ebf885bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:04.950420 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406167c5-6dfc-4606-9322-6e9ebf885bf1 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:04.950702 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406167c5-6dfc-4606-9322-6e9ebf885bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:04.950979 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406167c5-6dfc-4606-9322-6e9ebf885bf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:04.951699 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406167c5-6dfc-4606-9322-6e9ebf885bf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:04.953515 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406167c5-6dfc-4606-9322-6e9ebf885bf1 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:04.964748 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 927/1868] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:04 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:59:04.965634 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d75b0f-a545-471c-8a0f-a8ee2b61c0f0 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:04.980070 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 942/1869] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:04 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:59:04.980070 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406167c5-6dfc-4606-9322-6e9ebf885bf1 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:04.983800 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d75b0f-a545-471c-8a0f-a8ee2b61c0f0 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:04.989024 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f91a7e9-93ce-4e3f-bc4e-37c8c9d5bc4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:04.989864 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91a7e9-93ce-4e3f-bc4e-37c8c9d5bc4f admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:04.990163 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91a7e9-93ce-4e3f-bc4e-37c8c9d5bc4f admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:04.990441 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91a7e9-93ce-4e3f-bc4e-37c8c9d5bc4f admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:04.991310 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f91a7e9-93ce-4e3f-bc4e-37c8c9d5bc4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:04.994029 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406167c5-6dfc-4606-9322-6e9ebf885bf1 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:04.994196 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f91a7e9-93ce-4e3f-bc4e-37c8c9d5bc4f admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:05.000998 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52fbf55-b0c1-48dc-8aff-5a9e81bd27b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:05.001683 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52fbf55-b0c1-48dc-8aff-5a9e81bd27b2 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:05.001990 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52fbf55-b0c1-48dc-8aff-5a9e81bd27b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:05.002251 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52fbf55-b0c1-48dc-8aff-5a9e81bd27b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:05.002985 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52fbf55-b0c1-48dc-8aff-5a9e81bd27b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:05.005584 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52fbf55-b0c1-48dc-8aff-5a9e81bd27b2 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:05.031175 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 928/1871] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:59:04 2022] POST /identity/v3/projects => generated 385 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:59:05.031391 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 943/1871] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:04 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:59:05.032107 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f91a7e9-93ce-4e3f-bc4e-37c8c9d5bc4f admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:05.035143 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52fbf55-b0c1-48dc-8aff-5a9e81bd27b2 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:05.049306 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52fbf55-b0c1-48dc-8aff-5a9e81bd27b2 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:05.109486 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f91a7e9-93ce-4e3f-bc4e-37c8c9d5bc4f admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:05.116412 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f35e4a-eefc-4305-802f-2b00272fbe9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:05.117394 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f35e4a-eefc-4305-802f-2b00272fbe9a admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:05.117667 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f35e4a-eefc-4305-802f-2b00272fbe9a admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:05.118542 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f35e4a-eefc-4305-802f-2b00272fbe9a admin admin] PATH_INFO: `/v3/projects/b4e66868534f4df09eaa1e975f2890e1/users/a6932dd687284d78a5792c7b1d0ade8b/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:05.120786 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f35e4a-eefc-4305-802f-2b00272fbe9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4e66868534f4df09eaa1e975f2890e1, user_id=a6932dd687284d78a5792c7b1d0ade8b, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:05.123732 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f35e4a-eefc-4305-802f-2b00272fbe9a admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:05.124569 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06961db-66b9-482c-913d-e296bae4b4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:05.125733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06961db-66b9-482c-913d-e296bae4b4d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:05.126081 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06961db-66b9-482c-913d-e296bae4b4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:05.126376 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06961db-66b9-482c-913d-e296bae4b4d7 admin admin] PATH_INFO: `/v3/projects/cda4c4f83f1d48768d1131e3c4971fbf/users/75654ac027974a3db89a8c44d17f85df/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:05.135385 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 929/1872] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:59:05 2022] PUT /identity/v3/projects/b4e66868534f4df09eaa1e975f2890e1/users/a6932dd687284d78a5792c7b1d0ade8b/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:05.136366 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f35e4a-eefc-4305-802f-2b00272fbe9a admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:05.144258 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06961db-66b9-482c-913d-e296bae4b4d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cda4c4f83f1d48768d1131e3c4971fbf, user_id=75654ac027974a3db89a8c44d17f85df, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:05.147809 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06961db-66b9-482c-913d-e296bae4b4d7 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:05.174758 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 944/1873] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:59:05 2022] PUT /identity/v3/projects/cda4c4f83f1d48768d1131e3c4971fbf/users/75654ac027974a3db89a8c44d17f85df/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 14:59:05.176114 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa3876-7773-4ea8-a986-09bfa44958da None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:05.176397 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa3876-7773-4ea8-a986-09bfa44958da None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:05.176693 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa3876-7773-4ea8-a986-09bfa44958da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:05.205949 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-eaaa3876-7773-4ea8-a986-09bfa44958da None None] MFA Rules not processed for user `a6932dd687284d78a5792c7b1d0ade8b`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:05.207272 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-eaaa3876-7773-4ea8-a986-09bfa44958da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:05.209562 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f35e4a-eefc-4305-802f-2b00272fbe9a admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:05.269029 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a7cf4-45ee-481b-b198-c6ed4ac254d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:05.270516 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a7cf4-45ee-481b-b198-c6ed4ac254d6 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:05.270805 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a7cf4-45ee-481b-b198-c6ed4ac254d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:05.271083 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a7cf4-45ee-481b-b198-c6ed4ac254d6 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:05.271775 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631a7cf4-45ee-481b-b198-c6ed4ac254d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:05.273720 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631a7cf4-45ee-481b-b198-c6ed4ac254d6 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:05.291793 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 945/1874] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:05 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:05.295476 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628d0c6-2988-4e6c-a3ff-42c935040156 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:05.295476 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628d0c6-2988-4e6c-a3ff-42c935040156 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:05.295476 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628d0c6-2988-4e6c-a3ff-42c935040156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:05.299426 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 930/1875] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:59:05 2022] POST /identity/v3/users => generated 299 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:59:05.303289 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a7cf4-45ee-481b-b198-c6ed4ac254d6 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:05.315309 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a7cf4-45ee-481b-b198-c6ed4ac254d6 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:05.320379 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa27c0-538e-4481-903f-480ed24fb6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:05.321139 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa27c0-538e-4481-903f-480ed24fb6c0 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:05.321858 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa27c0-538e-4481-903f-480ed24fb6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:05.321858 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa27c0-538e-4481-903f-480ed24fb6c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:05.322747 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fa27c0-538e-4481-903f-480ed24fb6c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:05.324129 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-5628d0c6-2988-4e6c-a3ff-42c935040156 None None] MFA Rules not processed for user `75654ac027974a3db89a8c44d17f85df`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:05.324331 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fa27c0-538e-4481-903f-480ed24fb6c0 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:05.325371 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-5628d0c6-2988-4e6c-a3ff-42c935040156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:05.334951 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 931/1876] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:59:05.335573 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa27c0-538e-4481-903f-480ed24fb6c0 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:05.339500 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-18fa27c0-538e-4481-903f-480ed24fb6c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:05.401112 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa27c0-538e-4481-903f-480ed24fb6c0 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:05.408688 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db24bf4f-441f-45e6-a8b1-4e8ee53e34fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:05.409384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db24bf4f-441f-45e6-a8b1-4e8ee53e34fc service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:05.409664 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db24bf4f-441f-45e6-a8b1-4e8ee53e34fc service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:05.409939 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db24bf4f-441f-45e6-a8b1-4e8ee53e34fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:05.412412 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 946/1877] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:05 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:05.413610 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628d0c6-2988-4e6c-a3ff-42c935040156 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:05.428637 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628d0c6-2988-4e6c-a3ff-42c935040156 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:05.447899 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db24bf4f-441f-45e6-a8b1-4e8ee53e34fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:05.448007 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d7e94-4908-4ca6-baf5-eb5f258aea91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:05.448007 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d7e94-4908-4ca6-baf5-eb5f258aea91 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:05.448183 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db24bf4f-441f-45e6-a8b1-4e8ee53e34fc service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:05.448290 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d7e94-4908-4ca6-baf5-eb5f258aea91 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:05.448290 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d7e94-4908-4ca6-baf5-eb5f258aea91 admin admin] PATH_INFO: `/v3/projects/a26eb463d04e479a817558987b219290/users/8c02964e3ccf4f34bf5e1f2bd3962fc6/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:05.450323 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 932/1878] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:59:05 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:59:05.456440 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db24bf4f-441f-45e6-a8b1-4e8ee53e34fc service cinder] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:05.457603 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409d7e94-4908-4ca6-baf5-eb5f258aea91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a26eb463d04e479a817558987b219290, user_id=8c02964e3ccf4f34bf5e1f2bd3962fc6, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:05.458250 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-db24bf4f-441f-45e6-a8b1-4e8ee53e34fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:05.460276 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409d7e94-4908-4ca6-baf5-eb5f258aea91 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:05.471033 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db24bf4f-441f-45e6-a8b1-4e8ee53e34fc service cinder] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:05.480117 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088a087b-0cdb-422d-8d3e-926269f2b251 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:05.480813 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a087b-0cdb-422d-8d3e-926269f2b251 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:05.481218 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a087b-0cdb-422d-8d3e-926269f2b251 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:05.481500 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a087b-0cdb-422d-8d3e-926269f2b251 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:05.484495 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 947/1879] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:59:05 2022] PUT /identity/v3/projects/a26eb463d04e479a817558987b219290/users/8c02964e3ccf4f34bf5e1f2bd3962fc6/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:05.488703 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d7e94-4908-4ca6-baf5-eb5f258aea91 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:05.491113 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088a087b-0cdb-422d-8d3e-926269f2b251 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:05.493897 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088a087b-0cdb-422d-8d3e-926269f2b251 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:05.507318 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 933/1880] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:59:05 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 14:59:05.521787 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088a087b-0cdb-422d-8d3e-926269f2b251 service cinder] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:05.534998 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088a087b-0cdb-422d-8d3e-926269f2b251 service cinder] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:05.585204 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d7e94-4908-4ca6-baf5-eb5f258aea91 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:05.595047 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6f3d8f-ebbd-4f74-8b4b-a1e3400f0e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:05.595805 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f3d8f-ebbd-4f74-8b4b-a1e3400f0e78 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:05.596158 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f3d8f-ebbd-4f74-8b4b-a1e3400f0e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:05.596502 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f3d8f-ebbd-4f74-8b4b-a1e3400f0e78 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:05.597359 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6f3d8f-ebbd-4f74-8b4b-a1e3400f0e78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:05.599332 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6f3d8f-ebbd-4f74-8b4b-a1e3400f0e78 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:05.604123 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634dd852-b38a-4bd6-86e8-aeeb03d0d46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:05.605065 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634dd852-b38a-4bd6-86e8-aeeb03d0d46b admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:05.605345 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634dd852-b38a-4bd6-86e8-aeeb03d0d46b admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:05.605619 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634dd852-b38a-4bd6-86e8-aeeb03d0d46b admin admin] PATH_INFO: `/v3/users/75654ac027974a3db89a8c44d17f85df` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:05.606976 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634dd852-b38a-4bd6-86e8-aeeb03d0d46b admin admin] RBAC: Authorizing `identity:delete_user(user_id=75654ac027974a3db89a8c44d17f85df)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:05.608895 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634dd852-b38a-4bd6-86e8-aeeb03d0d46b admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:05.612780 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 948/1881] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:05 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:59:05.615714 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6f3d8f-ebbd-4f74-8b4b-a1e3400f0e78 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:05.626823 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6f3d8f-ebbd-4f74-8b4b-a1e3400f0e78 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:05.633434 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb847583-bef9-413f-ad8b-1ed7a762caad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:05.634729 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb847583-bef9-413f-ad8b-1ed7a762caad admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:05.635349 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb847583-bef9-413f-ad8b-1ed7a762caad admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:05.635770 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb847583-bef9-413f-ad8b-1ed7a762caad admin admin] PATH_INFO: `/v3/projects/a26eb463d04e479a817558987b219290/users/8c02964e3ccf4f34bf5e1f2bd3962fc6/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:05.638202 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb847583-bef9-413f-ad8b-1ed7a762caad admin admin] RBAC: Authorizing `identity:create_grant(project_id=a26eb463d04e479a817558987b219290, user_id=8c02964e3ccf4f34bf5e1f2bd3962fc6, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:05.642114 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb847583-bef9-413f-ad8b-1ed7a762caad admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:05.656863 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 949/1882] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:59:05 2022] PUT /identity/v3/projects/a26eb463d04e479a817558987b219290/users/8c02964e3ccf4f34bf5e1f2bd3962fc6/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 14:59:05.665525 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-634dd852-b38a-4bd6-86e8-aeeb03d0d46b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75654ac027974a3db89a8c44d17f85df'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:05.666500 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-634dd852-b38a-4bd6-86e8-aeeb03d0d46b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75654ac027974a3db89a8c44d17f85df'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:05.669351 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93e081-95f4-4345-ab07-3452a591d93a None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:05.669651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93e081-95f4-4345-ab07-3452a591d93a None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:05.669956 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93e081-95f4-4345-ab07-3452a591d93a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:05.675280 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-634dd852-b38a-4bd6-86e8-aeeb03d0d46b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75654ac027974a3db89a8c44d17f85df'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:05.683713 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-634dd852-b38a-4bd6-86e8-aeeb03d0d46b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75654ac027974a3db89a8c44d17f85df'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:05.684483 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-634dd852-b38a-4bd6-86e8-aeeb03d0d46b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75654ac027974a3db89a8c44d17f85df'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:05.714792 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 934/1883] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 14:59:05 2022] DELETE /identity/v3/users/75654ac027974a3db89a8c44d17f85df => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 14:59:05.723590 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-cb93e081-95f4-4345-ab07-3452a591d93a None None] MFA Rules not processed for user `8c02964e3ccf4f34bf5e1f2bd3962fc6`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:05.724950 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-cb93e081-95f4-4345-ab07-3452a591d93a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:05.730437 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d3569-8af8-4a59-ad75-b269a51dfa95 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:05.730726 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d3569-8af8-4a59-ad75-b269a51dfa95 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:05.731000 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d3569-8af8-4a59-ad75-b269a51dfa95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:05.750857 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-448d3569-8af8-4a59-ad75-b269a51dfa95 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:05.752112 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-448d3569-8af8-4a59-ad75-b269a51dfa95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:05.809751 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 950/1884] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:05 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:05.817918 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93e081-95f4-4345-ab07-3452a591d93a None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:05.819288 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-cb93e081-95f4-4345-ab07-3452a591d93a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:05.828181 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 935/1885] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:05.835664 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448d3569-8af8-4a59-ad75-b269a51dfa95 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:05.837197 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-448d3569-8af8-4a59-ad75-b269a51dfa95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:05.880055 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93e081-95f4-4345-ab07-3452a591d93a None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:05.884667 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd11bf-4f44-4efb-b26c-547e1722cf4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:05.885324 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd11bf-4f44-4efb-b26c-547e1722cf4c service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:05.885618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd11bf-4f44-4efb-b26c-547e1722cf4c service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:05.885951 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd11bf-4f44-4efb-b26c-547e1722cf4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:05.887998 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-3cfd11bf-4f44-4efb-b26c-547e1722cf4c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:05.901231 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448d3569-8af8-4a59-ad75-b269a51dfa95 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:05.905182 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76035ac4-309a-41bd-95c0-50051c85b821 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:05.906129 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76035ac4-309a-41bd-95c0-50051c85b821 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:05.906447 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76035ac4-309a-41bd-95c0-50051c85b821 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:05.906936 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76035ac4-309a-41bd-95c0-50051c85b821 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:05.917615 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76035ac4-309a-41bd-95c0-50051c85b821 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:05.919115 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76035ac4-309a-41bd-95c0-50051c85b821 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:05.929766 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 936/1886] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:59:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 14:59:05.956730 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfd11bf-4f44-4efb-b26c-547e1722cf4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:05.958248 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfd11bf-4f44-4efb-b26c-547e1722cf4c service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:05.969573 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 951/1887] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:59:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 14:59:06.207534 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76035ac4-309a-41bd-95c0-50051c85b821 service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:06.209049 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-76035ac4-309a-41bd-95c0-50051c85b821 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:06.231356 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76035ac4-309a-41bd-95c0-50051c85b821 service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:06.237097 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8861a5b-d3a0-409c-80b8-58e0677a9e3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:06.238576 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8861a5b-d3a0-409c-80b8-58e0677a9e3f service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:06.238992 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8861a5b-d3a0-409c-80b8-58e0677a9e3f service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:06.239279 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8861a5b-d3a0-409c-80b8-58e0677a9e3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:06.242145 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-c8861a5b-d3a0-409c-80b8-58e0677a9e3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:06.319804 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8861a5b-d3a0-409c-80b8-58e0677a9e3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:06.322637 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8861a5b-d3a0-409c-80b8-58e0677a9e3f service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:06.340048 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 937/1888] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:59:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:59:07.440266 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd11bf-4f44-4efb-b26c-547e1722cf4c service cinder] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:07.441758 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-3cfd11bf-4f44-4efb-b26c-547e1722cf4c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:07.474858 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd11bf-4f44-4efb-b26c-547e1722cf4c service cinder] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:07.482255 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991b59a-148a-4e4c-858e-18bccb9434a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:07.483134 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991b59a-148a-4e4c-858e-18bccb9434a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:07.483434 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991b59a-148a-4e4c-858e-18bccb9434a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:07.483799 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991b59a-148a-4e4c-858e-18bccb9434a7 admin admin] PATH_INFO: `/v3/projects/cda4c4f83f1d48768d1131e3c4971fbf` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:07.485668 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4991b59a-148a-4e4c-858e-18bccb9434a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cda4c4f83f1d48768d1131e3c4971fbf)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:07.487645 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4991b59a-148a-4e4c-858e-18bccb9434a7 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:07.541095 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-4991b59a-148a-4e4c-858e-18bccb9434a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cda4c4f83f1d48768d1131e3c4971fbf', 'request_id': 'req-4991b59a-148a-4e4c-858e-18bccb9434a7'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:07.546871 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-4991b59a-148a-4e4c-858e-18bccb9434a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cda4c4f83f1d48768d1131e3c4971fbf', 'request_id': 'req-4991b59a-148a-4e4c-858e-18bccb9434a7'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:07.555310 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-4991b59a-148a-4e4c-858e-18bccb9434a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cda4c4f83f1d48768d1131e3c4971fbf', 'request_id': 'req-4991b59a-148a-4e4c-858e-18bccb9434a7'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:07.568374 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-4991b59a-148a-4e4c-858e-18bccb9434a7 admin admin] The token cache is being invalidate because project cda4c4f83f1d48768d1131e3c4971fbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 14:59:07.568792 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-4991b59a-148a-4e4c-858e-18bccb9434a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:07.569594 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-4991b59a-148a-4e4c-858e-18bccb9434a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:07.571947 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 952/1889] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 14:59:07 2022] DELETE /identity/v3/projects/cda4c4f83f1d48768d1131e3c4971fbf => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:07.576622 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117ff79-5347-4732-a7ce-63bb765e33ca None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:07.577115 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117ff79-5347-4732-a7ce-63bb765e33ca None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:07.577197 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117ff79-5347-4732-a7ce-63bb765e33ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:07.595842 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-9117ff79-5347-4732-a7ce-63bb765e33ca None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:07.597239 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-9117ff79-5347-4732-a7ce-63bb765e33ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:07.677395 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9cb67-9505-4987-85d9-ebbd13adb76d None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:07.677687 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9cb67-9505-4987-85d9-ebbd13adb76d None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:07.677976 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9cb67-9505-4987-85d9-ebbd13adb76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:07.679359 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 938/1890] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:07.705285 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-7db9cb67-9505-4987-85d9-ebbd13adb76d None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:07.706522 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-7db9cb67-9505-4987-85d9-ebbd13adb76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:07.720843 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 953/1891] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 14:59:07.727774 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9117ff79-5347-4732-a7ce-63bb765e33ca None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:07.739502 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9117ff79-5347-4732-a7ce-63bb765e33ca None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:07.747128 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa323002-f6d3-4bf0-9c8b-01f16d67f5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:07.751049 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa323002-f6d3-4bf0-9c8b-01f16d67f5c9 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:07.751049 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa323002-f6d3-4bf0-9c8b-01f16d67f5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:07.751049 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa323002-f6d3-4bf0-9c8b-01f16d67f5c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:07.751049 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa323002-f6d3-4bf0-9c8b-01f16d67f5c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:59:07.751049 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa323002-f6d3-4bf0-9c8b-01f16d67f5c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:07.751049 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa323002-f6d3-4bf0-9c8b-01f16d67f5c9 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:07.774857 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 939/1892] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 14:59:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:59:07.780235 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db9cb67-9505-4987-85d9-ebbd13adb76d None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:07.803821 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db9cb67-9505-4987-85d9-ebbd13adb76d None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:07.810612 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d38bf4-11ba-4da5-b9a6-34bac10cec91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:07.811498 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d38bf4-11ba-4da5-b9a6-34bac10cec91 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:07.812127 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d38bf4-11ba-4da5-b9a6-34bac10cec91 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:07.812578 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d38bf4-11ba-4da5-b9a6-34bac10cec91 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:07.815149 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d38bf4-11ba-4da5-b9a6-34bac10cec91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:07.817356 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d38bf4-11ba-4da5-b9a6-34bac10cec91 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:07.859675 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 954/1893] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:59:07 2022] POST /identity/v3/projects => generated 373 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:59:07.860875 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa323002-f6d3-4bf0-9c8b-01f16d67f5c9 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:07.875490 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa323002-f6d3-4bf0-9c8b-01f16d67f5c9 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:08.001234 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054ecac-bd80-44b6-bf6f-1d9760b3fae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:08.002453 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054ecac-bd80-44b6-bf6f-1d9760b3fae2 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:08.002996 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054ecac-bd80-44b6-bf6f-1d9760b3fae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:08.003515 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054ecac-bd80-44b6-bf6f-1d9760b3fae2 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:08.004423 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e054ecac-bd80-44b6-bf6f-1d9760b3fae2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:08.007179 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e054ecac-bd80-44b6-bf6f-1d9760b3fae2 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:08.056977 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 940/1894] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:59:07 2022] POST /identity/v3/users => generated 287 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:59:08.057113 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d38bf4-11ba-4da5-b9a6-34bac10cec91 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:08.072234 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d38bf4-11ba-4da5-b9a6-34bac10cec91 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:08.077144 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee05424f-f590-4ac0-968b-60a393cdbf39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:08.077986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee05424f-f590-4ac0-968b-60a393cdbf39 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:08.078557 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee05424f-f590-4ac0-968b-60a393cdbf39 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:08.079005 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee05424f-f590-4ac0-968b-60a393cdbf39 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:08.079876 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee05424f-f590-4ac0-968b-60a393cdbf39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:08.081713 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee05424f-f590-4ac0-968b-60a393cdbf39 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:08.096409 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 955/1895] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:08 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:59:08.101943 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054ecac-bd80-44b6-bf6f-1d9760b3fae2 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:08.117538 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054ecac-bd80-44b6-bf6f-1d9760b3fae2 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:08.127071 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904e0e30-281b-4205-8628-a717ea739a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:08.128067 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e0e30-281b-4205-8628-a717ea739a1b admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:08.128683 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e0e30-281b-4205-8628-a717ea739a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:08.129139 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e0e30-281b-4205-8628-a717ea739a1b admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:08.130249 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904e0e30-281b-4205-8628-a717ea739a1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:08.132644 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904e0e30-281b-4205-8628-a717ea739a1b admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:08.145489 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 941/1896] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:08 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:59:08.156013 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee05424f-f590-4ac0-968b-60a393cdbf39 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:08.172533 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee05424f-f590-4ac0-968b-60a393cdbf39 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:08.186378 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc99195-1d22-4ca8-a0af-f1d4a8d23ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:08.187485 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc99195-1d22-4ca8-a0af-f1d4a8d23ea2 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:08.187903 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc99195-1d22-4ca8-a0af-f1d4a8d23ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:08.188187 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc99195-1d22-4ca8-a0af-f1d4a8d23ea2 admin admin] PATH_INFO: `/v3/projects/0124bfd61ad14bd395ecd3c54849f2fe/users/fdac43006996421799abf5bd401d0fc0/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:08.213467 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc99195-1d22-4ca8-a0af-f1d4a8d23ea2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0124bfd61ad14bd395ecd3c54849f2fe, user_id=fdac43006996421799abf5bd401d0fc0, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:08.216398 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc99195-1d22-4ca8-a0af-f1d4a8d23ea2 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:08.236937 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 956/1897] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:59:08 2022] PUT /identity/v3/projects/0124bfd61ad14bd395ecd3c54849f2fe/users/fdac43006996421799abf5bd401d0fc0/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:08.242165 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c4da4-8103-41b5-b67b-6f6e85751149 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:08.242474 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c4da4-8103-41b5-b67b-6f6e85751149 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:08.243024 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c4da4-8103-41b5-b67b-6f6e85751149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:08.285024 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-f64c4da4-8103-41b5-b67b-6f6e85751149 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:08.286326 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-f64c4da4-8103-41b5-b67b-6f6e85751149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:08.355183 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 942/1898] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:08.365154 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc99195-1d22-4ca8-a0af-f1d4a8d23ea2 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:08.366931 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-6dc99195-1d22-4ca8-a0af-f1d4a8d23ea2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:08.435519 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc99195-1d22-4ca8-a0af-f1d4a8d23ea2 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:08.439584 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c84e3-eb5f-4895-879d-20b566e20b6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:08.440239 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c84e3-eb5f-4895-879d-20b566e20b6d service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:08.440520 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c84e3-eb5f-4895-879d-20b566e20b6d service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:08.441099 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c84e3-eb5f-4895-879d-20b566e20b6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:08.450574 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88c84e3-eb5f-4895-879d-20b566e20b6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:08.452013 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88c84e3-eb5f-4895-879d-20b566e20b6d service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:08.464513 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 957/1899] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:59:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:59:09.884010 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad2aca-3ba6-4a7d-94df-79beaefabc2d None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:09.884995 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad2aca-3ba6-4a7d-94df-79beaefabc2d None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:09.885769 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad2aca-3ba6-4a7d-94df-79beaefabc2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:09.920185 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-b8ad2aca-3ba6-4a7d-94df-79beaefabc2d None None] MFA Rules not processed for user `fdac43006996421799abf5bd401d0fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:09.922369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b8ad2aca-3ba6-4a7d-94df-79beaefabc2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:10.015696 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 943/1900] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:10.028082 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c84e3-eb5f-4895-879d-20b566e20b6d service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:10.029974 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f88c84e3-eb5f-4895-879d-20b566e20b6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:10.109093 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c84e3-eb5f-4895-879d-20b566e20b6d service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:10.113066 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc3af2-dfc4-4877-a869-dc97209e6506 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:10.113726 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc3af2-dfc4-4877-a869-dc97209e6506 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:10.114008 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc3af2-dfc4-4877-a869-dc97209e6506 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:10.114288 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc3af2-dfc4-4877-a869-dc97209e6506 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:10.126663 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bc3af2-dfc4-4877-a869-dc97209e6506 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:10.128325 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bc3af2-dfc4-4877-a869-dc97209e6506 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:10.141081 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 958/1901] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:59:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:59:10.164386 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ad2aca-3ba6-4a7d-94df-79beaefabc2d None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:10.166264 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b8ad2aca-3ba6-4a7d-94df-79beaefabc2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:10.180746 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ad2aca-3ba6-4a7d-94df-79beaefabc2d None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:10.185994 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5784a1-7b5d-427d-bf25-fed618e187c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:10.187224 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5784a1-7b5d-427d-bf25-fed618e187c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:10.187742 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5784a1-7b5d-427d-bf25-fed618e187c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:10.188323 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5784a1-7b5d-427d-bf25-fed618e187c4 admin admin] PATH_INFO: `/v3/users/4c56619e7cb145f5930e46270571c828` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:10.189843 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5784a1-7b5d-427d-bf25-fed618e187c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c56619e7cb145f5930e46270571c828)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:10.192001 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5784a1-7b5d-427d-bf25-fed618e187c4 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:10.258260 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-dc5784a1-7b5d-427d-bf25-fed618e187c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c56619e7cb145f5930e46270571c828'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:10.260409 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-dc5784a1-7b5d-427d-bf25-fed618e187c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c56619e7cb145f5930e46270571c828'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:10.274873 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-dc5784a1-7b5d-427d-bf25-fed618e187c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c56619e7cb145f5930e46270571c828'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:10.287808 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-dc5784a1-7b5d-427d-bf25-fed618e187c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c56619e7cb145f5930e46270571c828'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:10.290521 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-dc5784a1-7b5d-427d-bf25-fed618e187c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c56619e7cb145f5930e46270571c828'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:10.325377 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 944/1902] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 14:59:10 2022] DELETE /identity/v3/users/4c56619e7cb145f5930e46270571c828 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:10.329836 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db3414d-1143-4b6a-9036-73a2675e20f0 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:10.330126 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db3414d-1143-4b6a-9036-73a2675e20f0 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:10.330654 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db3414d-1143-4b6a-9036-73a2675e20f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:10.356485 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-6db3414d-1143-4b6a-9036-73a2675e20f0 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:10.358464 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-6db3414d-1143-4b6a-9036-73a2675e20f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:10.475048 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 959/1903] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:10.486446 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5784a1-7b5d-427d-bf25-fed618e187c4 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:10.487904 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-dc5784a1-7b5d-427d-bf25-fed618e187c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:10.575190 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5784a1-7b5d-427d-bf25-fed618e187c4 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:10.581169 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79c0a1f-64ed-422e-92ab-72319d8c28d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:10.582215 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c0a1f-64ed-422e-92ab-72319d8c28d6 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:10.582773 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c0a1f-64ed-422e-92ab-72319d8c28d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:10.583179 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c0a1f-64ed-422e-92ab-72319d8c28d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:10.595259 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79c0a1f-64ed-422e-92ab-72319d8c28d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:10.597480 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79c0a1f-64ed-422e-92ab-72319d8c28d6 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:10.608748 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 945/1904] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:59:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:59:10.872658 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db3414d-1143-4b6a-9036-73a2675e20f0 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:10.874747 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-6db3414d-1143-4b6a-9036-73a2675e20f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:10.959450 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db3414d-1143-4b6a-9036-73a2675e20f0 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:10.965462 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2be832-6e15-424b-9a97-7eb541ce870e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:10.966200 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2be832-6e15-424b-9a97-7eb541ce870e service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:10.966548 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2be832-6e15-424b-9a97-7eb541ce870e service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:10.967141 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2be832-6e15-424b-9a97-7eb541ce870e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:10.968844 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-5a2be832-6e15-424b-9a97-7eb541ce870e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:10.989911 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79c0a1f-64ed-422e-92ab-72319d8c28d6 service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:10.991470 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-f79c0a1f-64ed-422e-92ab-72319d8c28d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:11.028581 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79c0a1f-64ed-422e-92ab-72319d8c28d6 service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:11.047889 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4e2fa-6add-4347-a9d8-21f4f62ab076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:11.050099 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4e2fa-6add-4347-a9d8-21f4f62ab076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:11.050401 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4e2fa-6add-4347-a9d8-21f4f62ab076 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:11.050788 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4e2fa-6add-4347-a9d8-21f4f62ab076 admin admin] PATH_INFO: `/v3/projects/30209268284f4cf791a88f4b623e4d1f` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:11.052503 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e4e2fa-6add-4347-a9d8-21f4f62ab076 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30209268284f4cf791a88f4b623e4d1f)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:11.056394 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e4e2fa-6add-4347-a9d8-21f4f62ab076 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:11.081755 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2be832-6e15-424b-9a97-7eb541ce870e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:11.083869 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2be832-6e15-424b-9a97-7eb541ce870e service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:11.105386 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 960/1905] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:59:10 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:59:11.123832 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-f2e4e2fa-6add-4347-a9d8-21f4f62ab076 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30209268284f4cf791a88f4b623e4d1f', 'request_id': 'req-f2e4e2fa-6add-4347-a9d8-21f4f62ab076'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:11.130440 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-f2e4e2fa-6add-4347-a9d8-21f4f62ab076 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30209268284f4cf791a88f4b623e4d1f', 'request_id': 'req-f2e4e2fa-6add-4347-a9d8-21f4f62ab076'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:11.142562 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-f2e4e2fa-6add-4347-a9d8-21f4f62ab076 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30209268284f4cf791a88f4b623e4d1f', 'request_id': 'req-f2e4e2fa-6add-4347-a9d8-21f4f62ab076'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:11.173905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-f2e4e2fa-6add-4347-a9d8-21f4f62ab076 admin admin] The token cache is being invalidate because project 30209268284f4cf791a88f4b623e4d1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 14:59:11.174242 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-f2e4e2fa-6add-4347-a9d8-21f4f62ab076 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:11.174922 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-f2e4e2fa-6add-4347-a9d8-21f4f62ab076 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:11.179996 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 946/1906] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 14:59:10 2022] DELETE /identity/v3/projects/30209268284f4cf791a88f4b623e4d1f => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:11.181416 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2be832-6e15-424b-9a97-7eb541ce870e service cinder] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:11.195163 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2be832-6e15-424b-9a97-7eb541ce870e service cinder] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:11.198579 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4e2fa-6add-4347-a9d8-21f4f62ab076 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:11.200918 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-f2e4e2fa-6add-4347-a9d8-21f4f62ab076 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:11.261170 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12612b9a-5119-433f-9251-92cb8e654b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:11.262086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12612b9a-5119-433f-9251-92cb8e654b05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:11.262370 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12612b9a-5119-433f-9251-92cb8e654b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:11.262678 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12612b9a-5119-433f-9251-92cb8e654b05 admin admin] PATH_INFO: `/v3/users/6336dc5822f34ac7bb02f8aa56d5c9c9` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:11.264440 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12612b9a-5119-433f-9251-92cb8e654b05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6336dc5822f34ac7bb02f8aa56d5c9c9)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:11.266461 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4e2fa-6add-4347-a9d8-21f4f62ab076 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:11.266589 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12612b9a-5119-433f-9251-92cb8e654b05 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:11.271991 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602a9a63-e137-430c-a0f0-f495b03a775c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:11.272724 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a9a63-e137-430c-a0f0-f495b03a775c service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:11.273049 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a9a63-e137-430c-a0f0-f495b03a775c service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:11.273333 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a9a63-e137-430c-a0f0-f495b03a775c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:11.275886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-602a9a63-e137-430c-a0f0-f495b03a775c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:11.329378 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-12612b9a-5119-433f-9251-92cb8e654b05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6336dc5822f34ac7bb02f8aa56d5c9c9'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:11.344829 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-12612b9a-5119-433f-9251-92cb8e654b05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6336dc5822f34ac7bb02f8aa56d5c9c9'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:11.348627 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602a9a63-e137-430c-a0f0-f495b03a775c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:11.350182 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602a9a63-e137-430c-a0f0-f495b03a775c service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:11.352478 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-12612b9a-5119-433f-9251-92cb8e654b05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6336dc5822f34ac7bb02f8aa56d5c9c9'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:11.353203 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-12612b9a-5119-433f-9251-92cb8e654b05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6336dc5822f34ac7bb02f8aa56d5c9c9'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:11.353989 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-12612b9a-5119-433f-9251-92cb8e654b05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6336dc5822f34ac7bb02f8aa56d5c9c9'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:11.360712 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 947/1907] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:59:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 14:59:11.372873 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 961/1908] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 14:59:11 2022] DELETE /identity/v3/users/6336dc5822f34ac7bb02f8aa56d5c9c9 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 14:59:11.736431 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602a9a63-e137-430c-a0f0-f495b03a775c service cinder] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:11.747389 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-602a9a63-e137-430c-a0f0-f495b03a775c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:11.768384 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcdcf14-e9fb-42b3-bb35-738b619f4492 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:11.768903 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcdcf14-e9fb-42b3-bb35-738b619f4492 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:11.769439 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcdcf14-e9fb-42b3-bb35-738b619f4492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:11.780926 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602a9a63-e137-430c-a0f0-f495b03a775c service cinder] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:11.839829 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-3dcdcf14-e9fb-42b3-bb35-738b619f4492 None None] MFA Rules not processed for user `fdac43006996421799abf5bd401d0fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:11.845029 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-3dcdcf14-e9fb-42b3-bb35-738b619f4492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:11.931668 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe8e4e-3540-45e2-9243-9df15040288d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:11.934771 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe8e4e-3540-45e2-9243-9df15040288d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:11.935807 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe8e4e-3540-45e2-9243-9df15040288d admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:11.936901 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe8e4e-3540-45e2-9243-9df15040288d admin admin] PATH_INFO: `/v3/projects/f0b842b03a8c4beda117c1710302aae0` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:11.940714 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe8e4e-3540-45e2-9243-9df15040288d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0b842b03a8c4beda117c1710302aae0)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:11.941269 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 962/1909] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:11.946536 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe8e4e-3540-45e2-9243-9df15040288d admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:12.052202 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-5bfe8e4e-3540-45e2-9243-9df15040288d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0b842b03a8c4beda117c1710302aae0', 'request_id': 'req-5bfe8e4e-3540-45e2-9243-9df15040288d'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:12.066572 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-5bfe8e4e-3540-45e2-9243-9df15040288d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0b842b03a8c4beda117c1710302aae0', 'request_id': 'req-5bfe8e4e-3540-45e2-9243-9df15040288d'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:12.089101 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-5bfe8e4e-3540-45e2-9243-9df15040288d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0b842b03a8c4beda117c1710302aae0', 'request_id': 'req-5bfe8e4e-3540-45e2-9243-9df15040288d'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:12.111137 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-5bfe8e4e-3540-45e2-9243-9df15040288d admin admin] The token cache is being invalidate because project f0b842b03a8c4beda117c1710302aae0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 14:59:12.111647 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-5bfe8e4e-3540-45e2-9243-9df15040288d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:12.114108 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-5bfe8e4e-3540-45e2-9243-9df15040288d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:12.117084 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 948/1910] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 14:59:11 2022] DELETE /identity/v3/projects/f0b842b03a8c4beda117c1710302aae0 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:12.124503 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab399f75-2c3a-48e6-a65d-072262194ea1 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:12.125459 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab399f75-2c3a-48e6-a65d-072262194ea1 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:12.125459 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab399f75-2c3a-48e6-a65d-072262194ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:12.145424 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-ab399f75-2c3a-48e6-a65d-072262194ea1 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:12.147116 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-ab399f75-2c3a-48e6-a65d-072262194ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:12.260835 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 963/1911] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:12.268809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca19960-4ed4-40a4-b0bd-06da3113280f None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:12.269111 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca19960-4ed4-40a4-b0bd-06da3113280f None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:12.269443 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca19960-4ed4-40a4-b0bd-06da3113280f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:12.295325 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-8ca19960-4ed4-40a4-b0bd-06da3113280f None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:12.297360 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-8ca19960-4ed4-40a4-b0bd-06da3113280f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:12.314591 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab399f75-2c3a-48e6-a65d-072262194ea1 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:12.315486 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 949/1912] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:12.327513 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab399f75-2c3a-48e6-a65d-072262194ea1 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:12.336498 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb859cb1-9280-4aa4-aeca-00b1c07b2dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:12.337931 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb859cb1-9280-4aa4-aeca-00b1c07b2dcb admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:12.339879 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb859cb1-9280-4aa4-aeca-00b1c07b2dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:12.340314 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb859cb1-9280-4aa4-aeca-00b1c07b2dcb admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:12.341231 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb859cb1-9280-4aa4-aeca-00b1c07b2dcb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:59:12.341709 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb859cb1-9280-4aa4-aeca-00b1c07b2dcb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:12.343520 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb859cb1-9280-4aa4-aeca-00b1c07b2dcb admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:12.363173 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 964/1913] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 14:59:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:59:12.364111 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca19960-4ed4-40a4-b0bd-06da3113280f None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:12.375136 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca19960-4ed4-40a4-b0bd-06da3113280f None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:12.379965 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9d5d1e-37c0-4be7-9e57-86a44d36d2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:12.380695 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d5d1e-37c0-4be7-9e57-86a44d36d2d1 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:12.380980 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d5d1e-37c0-4be7-9e57-86a44d36d2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:12.381253 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d5d1e-37c0-4be7-9e57-86a44d36d2d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:12.381967 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9d5d1e-37c0-4be7-9e57-86a44d36d2d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:12.384568 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9d5d1e-37c0-4be7-9e57-86a44d36d2d1 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:12.428653 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 950/1914] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:59:12 2022] POST /identity/v3/projects => generated 401 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 14:59:12.430880 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb859cb1-9280-4aa4-aeca-00b1c07b2dcb admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:12.442986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb859cb1-9280-4aa4-aeca-00b1c07b2dcb admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:12.546839 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbce20d-97ed-466b-89c5-337813e90b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:12.547559 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbce20d-97ed-466b-89c5-337813e90b0e admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:12.547848 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbce20d-97ed-466b-89c5-337813e90b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:12.548121 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbce20d-97ed-466b-89c5-337813e90b0e admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:12.549439 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbce20d-97ed-466b-89c5-337813e90b0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:12.554071 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbce20d-97ed-466b-89c5-337813e90b0e admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:12.597622 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 965/1915] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:59:12 2022] POST /identity/v3/users => generated 301 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 14:59:12.599082 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9d5d1e-37c0-4be7-9e57-86a44d36d2d1 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:12.615829 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9d5d1e-37c0-4be7-9e57-86a44d36d2d1 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:12.629201 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a90820-3f39-4a77-b61f-3293019458e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:12.631337 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a90820-3f39-4a77-b61f-3293019458e1 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:12.634040 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a90820-3f39-4a77-b61f-3293019458e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:12.634655 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a90820-3f39-4a77-b61f-3293019458e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:12.636397 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a90820-3f39-4a77-b61f-3293019458e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:12.639916 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a90820-3f39-4a77-b61f-3293019458e1 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:12.683973 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 951/1916] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:59:12.685377 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbce20d-97ed-466b-89c5-337813e90b0e admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:12.700183 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbce20d-97ed-466b-89c5-337813e90b0e admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:12.706762 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab9b00-cf97-4a33-a170-b851c93079b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:12.707546 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab9b00-cf97-4a33-a170-b851c93079b0 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:12.707837 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab9b00-cf97-4a33-a170-b851c93079b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:12.708117 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab9b00-cf97-4a33-a170-b851c93079b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:12.709004 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab9b00-cf97-4a33-a170-b851c93079b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:12.710663 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab9b00-cf97-4a33-a170-b851c93079b0 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:12.733535 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 966/1917] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:59:12.737973 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a90820-3f39-4a77-b61f-3293019458e1 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:12.754525 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a90820-3f39-4a77-b61f-3293019458e1 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:12.768466 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc613fcc-1407-495a-826f-3c3ad8bd5811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:12.770669 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc613fcc-1407-495a-826f-3c3ad8bd5811 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:12.771014 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc613fcc-1407-495a-826f-3c3ad8bd5811 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:12.771355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc613fcc-1407-495a-826f-3c3ad8bd5811 admin admin] PATH_INFO: `/v3/projects/f794f875ef854455b4bb9c5e2e1bd59f/users/e1a39c411fc5431da160f2293d69b480/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:12.792470 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc613fcc-1407-495a-826f-3c3ad8bd5811 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f794f875ef854455b4bb9c5e2e1bd59f, user_id=e1a39c411fc5431da160f2293d69b480, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:12.795991 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc613fcc-1407-495a-826f-3c3ad8bd5811 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:12.814583 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 952/1918] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:59:12 2022] PUT /identity/v3/projects/f794f875ef854455b4bb9c5e2e1bd59f/users/e1a39c411fc5431da160f2293d69b480/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 14:59:12.828142 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef05f2e-a9af-4515-ab69-b0e45deb74e1 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:12.828483 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef05f2e-a9af-4515-ab69-b0e45deb74e1 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:12.828969 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef05f2e-a9af-4515-ab69-b0e45deb74e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:12.870696 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-4ef05f2e-a9af-4515-ab69-b0e45deb74e1 None None] MFA Rules not processed for user `e1a39c411fc5431da160f2293d69b480`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:12.872279 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-4ef05f2e-a9af-4515-ab69-b0e45deb74e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:12.966525 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 967/1919] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:12 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:12.973681 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc613fcc-1407-495a-826f-3c3ad8bd5811 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:13.111746 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc613fcc-1407-495a-826f-3c3ad8bd5811 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:13.122806 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4036f47f-06f3-4b1c-8c7e-cb12d1ae41b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:13.125236 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036f47f-06f3-4b1c-8c7e-cb12d1ae41b1 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:13.125693 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036f47f-06f3-4b1c-8c7e-cb12d1ae41b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:13.126081 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036f47f-06f3-4b1c-8c7e-cb12d1ae41b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:13.127372 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4036f47f-06f3-4b1c-8c7e-cb12d1ae41b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:13.130868 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4036f47f-06f3-4b1c-8c7e-cb12d1ae41b1 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:13.192080 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 953/1920] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:59:12 2022] POST /identity/v3/projects => generated 401 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 14:59:13.196855 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef05f2e-a9af-4515-ab69-b0e45deb74e1 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:13.216752 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef05f2e-a9af-4515-ab69-b0e45deb74e1 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:13.328897 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07747ae6-fa1c-49b7-8065-578a6755c80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:13.329605 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07747ae6-fa1c-49b7-8065-578a6755c80b admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:13.329906 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07747ae6-fa1c-49b7-8065-578a6755c80b admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:13.335892 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07747ae6-fa1c-49b7-8065-578a6755c80b admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:13.335892 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07747ae6-fa1c-49b7-8065-578a6755c80b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:13.335892 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07747ae6-fa1c-49b7-8065-578a6755c80b admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:13.375201 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 968/1921] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:59:13 2022] POST /identity/v3/users => generated 307 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:59:13.379744 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4036f47f-06f3-4b1c-8c7e-cb12d1ae41b1 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:13.401695 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4036f47f-06f3-4b1c-8c7e-cb12d1ae41b1 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:13.424787 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d23ec-bc61-4a9f-bca6-ad910df62bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:13.426127 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d23ec-bc61-4a9f-bca6-ad910df62bbc admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:13.429177 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d23ec-bc61-4a9f-bca6-ad910df62bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:13.429651 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d23ec-bc61-4a9f-bca6-ad910df62bbc admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:13.439655 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d23ec-bc61-4a9f-bca6-ad910df62bbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:13.445195 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d23ec-bc61-4a9f-bca6-ad910df62bbc admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:13.472541 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 954/1922] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:13 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 06 14:59:13.476551 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07747ae6-fa1c-49b7-8065-578a6755c80b admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:13.496152 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07747ae6-fa1c-49b7-8065-578a6755c80b admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:13.503894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c12d17-e701-4726-8144-27db70e9f8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:13.505452 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c12d17-e701-4726-8144-27db70e9f8b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:13.505797 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c12d17-e701-4726-8144-27db70e9f8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:13.506081 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c12d17-e701-4726-8144-27db70e9f8b8 admin admin] PATH_INFO: `/v3/projects/ad416dc92ed1493caa1132d606b7cbd1/users/207bbaf61a804d178ce156dc7600e6b9/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:13.553886 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c12d17-e701-4726-8144-27db70e9f8b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad416dc92ed1493caa1132d606b7cbd1, user_id=207bbaf61a804d178ce156dc7600e6b9, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:13.565712 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c12d17-e701-4726-8144-27db70e9f8b8 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:13.594233 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 969/1923] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:59:13 2022] PUT /identity/v3/projects/ad416dc92ed1493caa1132d606b7cbd1/users/207bbaf61a804d178ce156dc7600e6b9/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:13.598100 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d23ec-bc61-4a9f-bca6-ad910df62bbc admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:13.613098 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d23ec-bc61-4a9f-bca6-ad910df62bbc admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:13.723203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be4c6c0-6f42-4c08-bdb3-0f11397369ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:13.723649 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4c6c0-6f42-4c08-bdb3-0f11397369ec admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:13.723940 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4c6c0-6f42-4c08-bdb3-0f11397369ec admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:13.724264 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4c6c0-6f42-4c08-bdb3-0f11397369ec admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:13.725160 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be4c6c0-6f42-4c08-bdb3-0f11397369ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:13.726830 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be4c6c0-6f42-4c08-bdb3-0f11397369ec admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:13.746924 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 955/1924] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:13 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:59:13.748098 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c12d17-e701-4726-8144-27db70e9f8b8 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:13.762617 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c12d17-e701-4726-8144-27db70e9f8b8 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:13.768448 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30da8bc-e6ce-491b-957c-75f5219302eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:13.769895 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30da8bc-e6ce-491b-957c-75f5219302eb admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:13.769981 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30da8bc-e6ce-491b-957c-75f5219302eb admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:13.770285 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30da8bc-e6ce-491b-957c-75f5219302eb admin admin] PATH_INFO: `/v3/projects/ad416dc92ed1493caa1132d606b7cbd1/users/207bbaf61a804d178ce156dc7600e6b9/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:13.772279 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30da8bc-e6ce-491b-957c-75f5219302eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad416dc92ed1493caa1132d606b7cbd1, user_id=207bbaf61a804d178ce156dc7600e6b9, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:13.775788 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30da8bc-e6ce-491b-957c-75f5219302eb admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:13.788961 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 970/1925] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:59:13 2022] PUT /identity/v3/projects/ad416dc92ed1493caa1132d606b7cbd1/users/207bbaf61a804d178ce156dc7600e6b9/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:13.800870 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4ab41-56d9-4043-b57e-4125973c3047 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:13.801202 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4ab41-56d9-4043-b57e-4125973c3047 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:13.801516 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4ab41-56d9-4043-b57e-4125973c3047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:13.834004 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-37b4ab41-56d9-4043-b57e-4125973c3047 None None] MFA Rules not processed for user `207bbaf61a804d178ce156dc7600e6b9`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:13.836059 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-37b4ab41-56d9-4043-b57e-4125973c3047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:13.919112 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 956/1926] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:13 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:13.937794 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30da8bc-e6ce-491b-957c-75f5219302eb admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:13.939532 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-d30da8bc-e6ce-491b-957c-75f5219302eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:14.011745 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30da8bc-e6ce-491b-957c-75f5219302eb admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:14.015884 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d5e387-148b-4a55-9ece-2fe45595bc4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:14.016559 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5e387-148b-4a55-9ece-2fe45595bc4d service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:14.016957 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5e387-148b-4a55-9ece-2fe45595bc4d service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:14.017191 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5e387-148b-4a55-9ece-2fe45595bc4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:14.028787 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d5e387-148b-4a55-9ece-2fe45595bc4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:14.030235 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d5e387-148b-4a55-9ece-2fe45595bc4d service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:14.042801 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 971/1927] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:59:13 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:59:14.108312 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b4ab41-56d9-4043-b57e-4125973c3047 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:14.110369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-37b4ab41-56d9-4043-b57e-4125973c3047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:14.161046 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d5e387-148b-4a55-9ece-2fe45595bc4d service cinder] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:14.174833 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d5e387-148b-4a55-9ece-2fe45595bc4d service cinder] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:14.179556 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652be19f-7105-4f1e-852d-00c0b89bf2c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:14.181738 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652be19f-7105-4f1e-852d-00c0b89bf2c7 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:14.181738 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652be19f-7105-4f1e-852d-00c0b89bf2c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:14.181738 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652be19f-7105-4f1e-852d-00c0b89bf2c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:14.228942 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b4ab41-56d9-4043-b57e-4125973c3047 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:14.237828 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8348644-ee60-491e-aa8c-811e9f31bbc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:14.239240 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8348644-ee60-491e-aa8c-811e9f31bbc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:14.239944 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8348644-ee60-491e-aa8c-811e9f31bbc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:14.240423 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8348644-ee60-491e-aa8c-811e9f31bbc0 admin admin] PATH_INFO: `/v3/users/590985dde4fa4defb550529858e16ce2` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:14.243883 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8348644-ee60-491e-aa8c-811e9f31bbc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=590985dde4fa4defb550529858e16ce2)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:14.247708 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8348644-ee60-491e-aa8c-811e9f31bbc0 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:14.248254 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652be19f-7105-4f1e-852d-00c0b89bf2c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:14.249944 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652be19f-7105-4f1e-852d-00c0b89bf2c7 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:14.262644 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 972/1928] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:59:14 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 14:59:14.322037 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-f8348644-ee60-491e-aa8c-811e9f31bbc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '590985dde4fa4defb550529858e16ce2'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:14.323346 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-f8348644-ee60-491e-aa8c-811e9f31bbc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '590985dde4fa4defb550529858e16ce2'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:14.334952 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-f8348644-ee60-491e-aa8c-811e9f31bbc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '590985dde4fa4defb550529858e16ce2'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:14.346441 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-f8348644-ee60-491e-aa8c-811e9f31bbc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '590985dde4fa4defb550529858e16ce2'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:14.347322 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-f8348644-ee60-491e-aa8c-811e9f31bbc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '590985dde4fa4defb550529858e16ce2'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:14.381736 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 957/1929] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 14:59:14 2022] DELETE /identity/v3/users/590985dde4fa4defb550529858e16ce2 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:14.386172 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005c114-8ca3-4c4f-8c5d-3f8c5091d3a9 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:14.386456 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005c114-8ca3-4c4f-8c5d-3f8c5091d3a9 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:14.386884 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005c114-8ca3-4c4f-8c5d-3f8c5091d3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:14.409354 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-0005c114-8ca3-4c4f-8c5d-3f8c5091d3a9 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:14.410751 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-0005c114-8ca3-4c4f-8c5d-3f8c5091d3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:14.503256 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 973/1930] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:14.520659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8348644-ee60-491e-aa8c-811e9f31bbc0 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:14.520659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-f8348644-ee60-491e-aa8c-811e9f31bbc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:14.598510 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8348644-ee60-491e-aa8c-811e9f31bbc0 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:14.604509 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc867a-b2af-436e-b3d3-146375a388fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:14.605293 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc867a-b2af-436e-b3d3-146375a388fc service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:14.605575 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc867a-b2af-436e-b3d3-146375a388fc service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:14.605850 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc867a-b2af-436e-b3d3-146375a388fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:14.618658 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dc867a-b2af-436e-b3d3-146375a388fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:14.620137 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dc867a-b2af-436e-b3d3-146375a388fc service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:14.632201 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 958/1931] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:59:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:59:15.312226 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005c114-8ca3-4c4f-8c5d-3f8c5091d3a9 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:15.314098 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-0005c114-8ca3-4c4f-8c5d-3f8c5091d3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:15.335895 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005c114-8ca3-4c4f-8c5d-3f8c5091d3a9 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:15.356575 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00737ff6-79f2-4542-8b5a-557fff34fb04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:15.357534 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00737ff6-79f2-4542-8b5a-557fff34fb04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:15.357822 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00737ff6-79f2-4542-8b5a-557fff34fb04 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:15.358105 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00737ff6-79f2-4542-8b5a-557fff34fb04 admin admin] PATH_INFO: `/v3/projects/7556b262575047aca552e72d6828468d` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:15.366565 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00737ff6-79f2-4542-8b5a-557fff34fb04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7556b262575047aca552e72d6828468d)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:15.376083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00737ff6-79f2-4542-8b5a-557fff34fb04 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:15.455324 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-00737ff6-79f2-4542-8b5a-557fff34fb04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7556b262575047aca552e72d6828468d', 'request_id': 'req-00737ff6-79f2-4542-8b5a-557fff34fb04'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:15.462490 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-00737ff6-79f2-4542-8b5a-557fff34fb04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7556b262575047aca552e72d6828468d', 'request_id': 'req-00737ff6-79f2-4542-8b5a-557fff34fb04'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:15.470623 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-00737ff6-79f2-4542-8b5a-557fff34fb04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7556b262575047aca552e72d6828468d', 'request_id': 'req-00737ff6-79f2-4542-8b5a-557fff34fb04'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:15.483411 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-00737ff6-79f2-4542-8b5a-557fff34fb04 admin admin] The token cache is being invalidate because project 7556b262575047aca552e72d6828468d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 14:59:15.483940 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-00737ff6-79f2-4542-8b5a-557fff34fb04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:15.484656 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-00737ff6-79f2-4542-8b5a-557fff34fb04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:15.486247 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 974/1932] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 14:59:15 2022] DELETE /identity/v3/projects/7556b262575047aca552e72d6828468d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:15.490524 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc867a-b2af-436e-b3d3-146375a388fc service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:15.502148 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc867a-b2af-436e-b3d3-146375a388fc service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:15.576312 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8753bcab-856b-4ff0-80b3-6fd978b2e14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:15.577507 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8753bcab-856b-4ff0-80b3-6fd978b2e14d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:15.577898 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8753bcab-856b-4ff0-80b3-6fd978b2e14d admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:15.578411 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8753bcab-856b-4ff0-80b3-6fd978b2e14d admin admin] PATH_INFO: `/v3/users/d78e95c546ae4f8584bcacace9b2ce8c` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:15.579887 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8753bcab-856b-4ff0-80b3-6fd978b2e14d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d78e95c546ae4f8584bcacace9b2ce8c)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:15.581942 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8753bcab-856b-4ff0-80b3-6fd978b2e14d admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:15.631002 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8753bcab-856b-4ff0-80b3-6fd978b2e14d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd78e95c546ae4f8584bcacace9b2ce8c'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:15.631930 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8753bcab-856b-4ff0-80b3-6fd978b2e14d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd78e95c546ae4f8584bcacace9b2ce8c'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:15.640071 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8753bcab-856b-4ff0-80b3-6fd978b2e14d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd78e95c546ae4f8584bcacace9b2ce8c'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:15.646963 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8753bcab-856b-4ff0-80b3-6fd978b2e14d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd78e95c546ae4f8584bcacace9b2ce8c'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:15.647614 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8753bcab-856b-4ff0-80b3-6fd978b2e14d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd78e95c546ae4f8584bcacace9b2ce8c'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:15.665803 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 959/1933] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 14:59:15 2022] DELETE /identity/v3/users/d78e95c546ae4f8584bcacace9b2ce8c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:16.322691 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00737ff6-79f2-4542-8b5a-557fff34fb04 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:16.324310 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-00737ff6-79f2-4542-8b5a-557fff34fb04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:16.338629 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00737ff6-79f2-4542-8b5a-557fff34fb04 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:16.411034 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8753bcab-856b-4ff0-80b3-6fd978b2e14d admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:16.412540 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-8753bcab-856b-4ff0-80b3-6fd978b2e14d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:16.441769 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec09b88-7902-463f-868f-afebc26561b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:16.443157 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec09b88-7902-463f-868f-afebc26561b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:16.443611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec09b88-7902-463f-868f-afebc26561b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:16.444064 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec09b88-7902-463f-868f-afebc26561b4 admin admin] PATH_INFO: `/v3/projects/5011f1a7114f49678c6da0756ce0a3f0` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:16.445801 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec09b88-7902-463f-868f-afebc26561b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5011f1a7114f49678c6da0756ce0a3f0)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:16.448761 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec09b88-7902-463f-868f-afebc26561b4 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:16.519488 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-eec09b88-7902-463f-868f-afebc26561b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5011f1a7114f49678c6da0756ce0a3f0', 'request_id': 'req-eec09b88-7902-463f-868f-afebc26561b4'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:16.527149 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8753bcab-856b-4ff0-80b3-6fd978b2e14d admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:16.528525 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-eec09b88-7902-463f-868f-afebc26561b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5011f1a7114f49678c6da0756ce0a3f0', 'request_id': 'req-eec09b88-7902-463f-868f-afebc26561b4'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:16.538914 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-eec09b88-7902-463f-868f-afebc26561b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5011f1a7114f49678c6da0756ce0a3f0', 'request_id': 'req-eec09b88-7902-463f-868f-afebc26561b4'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:16.558221 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-eec09b88-7902-463f-868f-afebc26561b4 admin admin] The token cache is being invalidate because project 5011f1a7114f49678c6da0756ce0a3f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 14:59:16.558679 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-eec09b88-7902-463f-868f-afebc26561b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:16.559556 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-eec09b88-7902-463f-868f-afebc26561b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:16.562563 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 975/1934] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 14:59:16 2022] DELETE /identity/v3/projects/5011f1a7114f49678c6da0756ce0a3f0 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:16.569208 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec09b88-7902-463f-868f-afebc26561b4 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:16.574040 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-eec09b88-7902-463f-868f-afebc26561b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:16.595757 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68964638-8faa-43be-8b06-e3deab269419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:16.596794 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68964638-8faa-43be-8b06-e3deab269419 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:16.597167 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68964638-8faa-43be-8b06-e3deab269419 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:16.597556 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68964638-8faa-43be-8b06-e3deab269419 admin admin] PATH_INFO: `/v3/users/1d6b2f1beece46869291ed5d4fa5f6e9` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:16.599923 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68964638-8faa-43be-8b06-e3deab269419 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d6b2f1beece46869291ed5d4fa5f6e9)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:16.602405 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68964638-8faa-43be-8b06-e3deab269419 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:16.649579 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec09b88-7902-463f-868f-afebc26561b4 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:16.657503 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c11113-b3dd-40b8-91a1-cc0b2bc0887f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:16.658623 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c11113-b3dd-40b8-91a1-cc0b2bc0887f admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:16.659105 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c11113-b3dd-40b8-91a1-cc0b2bc0887f admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:16.659608 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c11113-b3dd-40b8-91a1-cc0b2bc0887f admin admin] PATH_INFO: `/v3/users/a6932dd687284d78a5792c7b1d0ade8b` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:16.661218 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c11113-b3dd-40b8-91a1-cc0b2bc0887f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6932dd687284d78a5792c7b1d0ade8b)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:16.663538 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c11113-b3dd-40b8-91a1-cc0b2bc0887f admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:16.673964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-68964638-8faa-43be-8b06-e3deab269419 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d6b2f1beece46869291ed5d4fa5f6e9'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:16.674875 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-68964638-8faa-43be-8b06-e3deab269419 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d6b2f1beece46869291ed5d4fa5f6e9'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:16.684749 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-68964638-8faa-43be-8b06-e3deab269419 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d6b2f1beece46869291ed5d4fa5f6e9'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:16.694824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-68964638-8faa-43be-8b06-e3deab269419 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d6b2f1beece46869291ed5d4fa5f6e9'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:16.695490 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-68964638-8faa-43be-8b06-e3deab269419 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d6b2f1beece46869291ed5d4fa5f6e9'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:16.716185 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 960/1935] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 14:59:16 2022] DELETE /identity/v3/users/1d6b2f1beece46869291ed5d4fa5f6e9 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:16.719168 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-80c11113-b3dd-40b8-91a1-cc0b2bc0887f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6932dd687284d78a5792c7b1d0ade8b'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:16.720318 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8f5aa-9e30-4127-aea8-c48ca8df40d2 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:16.720692 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8f5aa-9e30-4127-aea8-c48ca8df40d2 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:16.720976 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8f5aa-9e30-4127-aea8-c48ca8df40d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:16.728342 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-80c11113-b3dd-40b8-91a1-cc0b2bc0887f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6932dd687284d78a5792c7b1d0ade8b'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:16.735409 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-80c11113-b3dd-40b8-91a1-cc0b2bc0887f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6932dd687284d78a5792c7b1d0ade8b'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:16.736130 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-80c11113-b3dd-40b8-91a1-cc0b2bc0887f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6932dd687284d78a5792c7b1d0ade8b'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:16.736807 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-80c11113-b3dd-40b8-91a1-cc0b2bc0887f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6932dd687284d78a5792c7b1d0ade8b'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:16.740319 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-8ca8f5aa-9e30-4127-aea8-c48ca8df40d2 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:16.741556 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-8ca8f5aa-9e30-4127-aea8-c48ca8df40d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:16.755112 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 976/1936] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 14:59:16 2022] DELETE /identity/v3/users/a6932dd687284d78a5792c7b1d0ade8b => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:16.807038 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 961/1937] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 14:59:16.820577 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c11113-b3dd-40b8-91a1-cc0b2bc0887f admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:16.822749 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-80c11113-b3dd-40b8-91a1-cc0b2bc0887f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:16.921315 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c11113-b3dd-40b8-91a1-cc0b2bc0887f admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:16.926844 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ffa83-3bfa-4c00-a75d-7756c8383274 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:16.927634 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ffa83-3bfa-4c00-a75d-7756c8383274 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:16.928014 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ffa83-3bfa-4c00-a75d-7756c8383274 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:16.928388 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ffa83-3bfa-4c00-a75d-7756c8383274 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:17.026200 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ffa83-3bfa-4c00-a75d-7756c8383274 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:17.027761 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ffa83-3bfa-4c00-a75d-7756c8383274 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:17.038835 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 977/1938] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:59:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:59:17.349256 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8f5aa-9e30-4127-aea8-c48ca8df40d2 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:17.350270 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-8ca8f5aa-9e30-4127-aea8-c48ca8df40d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:17.372739 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8f5aa-9e30-4127-aea8-c48ca8df40d2 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:17.391572 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ba9af4-2814-4903-80c0-0ea414019f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:17.392621 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba9af4-2814-4903-80c0-0ea414019f4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:17.392983 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba9af4-2814-4903-80c0-0ea414019f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:17.393325 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba9af4-2814-4903-80c0-0ea414019f4b admin admin] PATH_INFO: `/v3/projects/b4e66868534f4df09eaa1e975f2890e1` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:17.394669 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ba9af4-2814-4903-80c0-0ea414019f4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4e66868534f4df09eaa1e975f2890e1)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:17.396709 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ba9af4-2814-4903-80c0-0ea414019f4b admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:17.462103 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-63ba9af4-2814-4903-80c0-0ea414019f4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4e66868534f4df09eaa1e975f2890e1', 'request_id': 'req-63ba9af4-2814-4903-80c0-0ea414019f4b'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:17.510072 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-63ba9af4-2814-4903-80c0-0ea414019f4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4e66868534f4df09eaa1e975f2890e1', 'request_id': 'req-63ba9af4-2814-4903-80c0-0ea414019f4b'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:17.530010 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-63ba9af4-2814-4903-80c0-0ea414019f4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4e66868534f4df09eaa1e975f2890e1', 'request_id': 'req-63ba9af4-2814-4903-80c0-0ea414019f4b'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:17.543589 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-63ba9af4-2814-4903-80c0-0ea414019f4b admin admin] The token cache is being invalidate because project b4e66868534f4df09eaa1e975f2890e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 14:59:17.544156 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-63ba9af4-2814-4903-80c0-0ea414019f4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:17.544706 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-63ba9af4-2814-4903-80c0-0ea414019f4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:17.546442 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 962/1939] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 14:59:17 2022] DELETE /identity/v3/projects/b4e66868534f4df09eaa1e975f2890e1 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:17.552574 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b4308-cf5e-465f-8afa-7545ffd1a9f5 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:17.553161 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b4308-cf5e-465f-8afa-7545ffd1a9f5 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:17.553245 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b4308-cf5e-465f-8afa-7545ffd1a9f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:17.573069 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-284b4308-cf5e-465f-8afa-7545ffd1a9f5 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:17.574436 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-284b4308-cf5e-465f-8afa-7545ffd1a9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:17.644305 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ba9af4-2814-4903-80c0-0ea414019f4b admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:17.646054 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-63ba9af4-2814-4903-80c0-0ea414019f4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:17.653288 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 978/1940] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:17.662702 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b6b065-a7a3-4ecc-8316-d105b6ff4f9f None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:17.662959 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b6b065-a7a3-4ecc-8316-d105b6ff4f9f None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:17.663087 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ba9af4-2814-4903-80c0-0ea414019f4b admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:17.663247 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b6b065-a7a3-4ecc-8316-d105b6ff4f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:17.669347 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ba050-3787-4a60-8688-da8b74317af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:17.670797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ba050-3787-4a60-8688-da8b74317af8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:17.671171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ba050-3787-4a60-8688-da8b74317af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:17.672481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ba050-3787-4a60-8688-da8b74317af8 admin admin] PATH_INFO: `/v3/projects/72ed39139a59446fa07d5e8812ee0d41` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:17.673405 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006ba050-3787-4a60-8688-da8b74317af8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72ed39139a59446fa07d5e8812ee0d41)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:17.675693 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006ba050-3787-4a60-8688-da8b74317af8 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:17.683061 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-04b6b065-a7a3-4ecc-8316-d105b6ff4f9f None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:17.684285 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-04b6b065-a7a3-4ecc-8316-d105b6ff4f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:17.692418 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 979/1941] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 14:59:17.696753 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b6b065-a7a3-4ecc-8316-d105b6ff4f9f None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:17.706704 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b6b065-a7a3-4ecc-8316-d105b6ff4f9f None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:17.713346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32acb12d-3524-49b2-a5fb-b9596709332a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:17.714012 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32acb12d-3524-49b2-a5fb-b9596709332a admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:17.714291 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32acb12d-3524-49b2-a5fb-b9596709332a admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:17.714567 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32acb12d-3524-49b2-a5fb-b9596709332a admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:17.715289 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32acb12d-3524-49b2-a5fb-b9596709332a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:59:17.715596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32acb12d-3524-49b2-a5fb-b9596709332a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:17.717217 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32acb12d-3524-49b2-a5fb-b9596709332a admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:17.727646 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-006ba050-3787-4a60-8688-da8b74317af8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72ed39139a59446fa07d5e8812ee0d41', 'request_id': 'req-006ba050-3787-4a60-8688-da8b74317af8'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:17.732798 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 980/1942] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 14:59:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:59:17.734399 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-006ba050-3787-4a60-8688-da8b74317af8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72ed39139a59446fa07d5e8812ee0d41', 'request_id': 'req-006ba050-3787-4a60-8688-da8b74317af8'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:17.735188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32acb12d-3524-49b2-a5fb-b9596709332a admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:17.743155 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-006ba050-3787-4a60-8688-da8b74317af8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72ed39139a59446fa07d5e8812ee0d41', 'request_id': 'req-006ba050-3787-4a60-8688-da8b74317af8'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:17.746002 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32acb12d-3524-49b2-a5fb-b9596709332a admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:17.755241 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-006ba050-3787-4a60-8688-da8b74317af8 admin admin] The token cache is being invalidate because project 72ed39139a59446fa07d5e8812ee0d41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 14:59:17.755427 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-006ba050-3787-4a60-8688-da8b74317af8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:17.756454 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-006ba050-3787-4a60-8688-da8b74317af8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:17.757889 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 963/1943] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 14:59:17 2022] DELETE /identity/v3/projects/72ed39139a59446fa07d5e8812ee0d41 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:17.761856 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ba050-3787-4a60-8688-da8b74317af8 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:17.763243 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-006ba050-3787-4a60-8688-da8b74317af8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:17.811088 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff6c3df-a811-48b7-b949-a981b9285dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:17.812107 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6c3df-a811-48b7-b949-a981b9285dc2 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:17.812397 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6c3df-a811-48b7-b949-a981b9285dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:17.813210 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6c3df-a811-48b7-b949-a981b9285dc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:17.814088 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff6c3df-a811-48b7-b949-a981b9285dc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:17.816021 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff6c3df-a811-48b7-b949-a981b9285dc2 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:17.834025 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ba050-3787-4a60-8688-da8b74317af8 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:17.839012 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb917e-5a73-4361-b6af-ea1903a60b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:17.839899 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb917e-5a73-4361-b6af-ea1903a60b64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:17.840180 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb917e-5a73-4361-b6af-ea1903a60b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:17.840461 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb917e-5a73-4361-b6af-ea1903a60b64 admin admin] PATH_INFO: `/v3/users/8c02964e3ccf4f34bf5e1f2bd3962fc6` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:17.841723 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bb917e-5a73-4361-b6af-ea1903a60b64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c02964e3ccf4f34bf5e1f2bd3962fc6)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:17.843658 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bb917e-5a73-4361-b6af-ea1903a60b64 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:17.845009 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 981/1944] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:59:17 2022] POST /identity/v3/projects => generated 391 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 14:59:17.849125 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff6c3df-a811-48b7-b949-a981b9285dc2 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:17.850769 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-1ff6c3df-a811-48b7-b949-a981b9285dc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:17.894473 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-b5bb917e-5a73-4361-b6af-ea1903a60b64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c02964e3ccf4f34bf5e1f2bd3962fc6'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:17.895401 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-b5bb917e-5a73-4361-b6af-ea1903a60b64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c02964e3ccf4f34bf5e1f2bd3962fc6'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:17.903861 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-b5bb917e-5a73-4361-b6af-ea1903a60b64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c02964e3ccf4f34bf5e1f2bd3962fc6'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:17.913419 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-b5bb917e-5a73-4361-b6af-ea1903a60b64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c02964e3ccf4f34bf5e1f2bd3962fc6'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:17.915083 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-b5bb917e-5a73-4361-b6af-ea1903a60b64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c02964e3ccf4f34bf5e1f2bd3962fc6'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:17.934208 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff6c3df-a811-48b7-b949-a981b9285dc2 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:17.937935 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 964/1945] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 14:59:17 2022] DELETE /identity/v3/users/8c02964e3ccf4f34bf5e1f2bd3962fc6 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:18.014254 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a3924-cfd0-4c14-a382-8b4124d2be93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:18.015275 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a3924-cfd0-4c14-a382-8b4124d2be93 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:18.015640 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a3924-cfd0-4c14-a382-8b4124d2be93 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:18.016025 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a3924-cfd0-4c14-a382-8b4124d2be93 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:18.016829 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6a3924-cfd0-4c14-a382-8b4124d2be93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:18.019620 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6a3924-cfd0-4c14-a382-8b4124d2be93 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:18.053482 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 982/1946] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:59:17 2022] POST /identity/v3/users => generated 296 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:59:18.058270 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb917e-5a73-4361-b6af-ea1903a60b64 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:18.059819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b5bb917e-5a73-4361-b6af-ea1903a60b64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:18.078996 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb917e-5a73-4361-b6af-ea1903a60b64 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:18.095526 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b9ef29-7a72-4512-bee7-b51eb0c04c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:18.096414 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b9ef29-7a72-4512-bee7-b51eb0c04c63 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:18.097130 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b9ef29-7a72-4512-bee7-b51eb0c04c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:18.097130 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b9ef29-7a72-4512-bee7-b51eb0c04c63 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:18.097873 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b9ef29-7a72-4512-bee7-b51eb0c04c63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:18.099632 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b9ef29-7a72-4512-bee7-b51eb0c04c63 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:18.120560 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 965/1947] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:59:18.125061 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a3924-cfd0-4c14-a382-8b4124d2be93 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:18.142238 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a3924-cfd0-4c14-a382-8b4124d2be93 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:18.151026 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7785051-618b-4418-a5c1-cdb23e4ee8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:18.151960 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7785051-618b-4418-a5c1-cdb23e4ee8e3 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:18.152474 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7785051-618b-4418-a5c1-cdb23e4ee8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:18.152958 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7785051-618b-4418-a5c1-cdb23e4ee8e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:18.154560 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7785051-618b-4418-a5c1-cdb23e4ee8e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:18.156763 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7785051-618b-4418-a5c1-cdb23e4ee8e3 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:18.172588 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 983/1948] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:59:18.177793 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b9ef29-7a72-4512-bee7-b51eb0c04c63 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:18.213896 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b9ef29-7a72-4512-bee7-b51eb0c04c63 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:18.220525 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810b44b0-b37c-4e09-a707-7d7f6b207d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:18.221461 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b44b0-b37c-4e09-a707-7d7f6b207d4b admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:18.221745 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b44b0-b37c-4e09-a707-7d7f6b207d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:18.222692 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b44b0-b37c-4e09-a707-7d7f6b207d4b admin admin] PATH_INFO: `/v3/projects/192b34a3662e4360b8e8450ad7b945e8/users/5d9b0e95b2e841739be689bcbe6d36a8/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:18.245136 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810b44b0-b37c-4e09-a707-7d7f6b207d4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=192b34a3662e4360b8e8450ad7b945e8, user_id=5d9b0e95b2e841739be689bcbe6d36a8, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:18.267513 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810b44b0-b37c-4e09-a707-7d7f6b207d4b admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:18.306990 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 966/1949] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:59:18 2022] PUT /identity/v3/projects/192b34a3662e4360b8e8450ad7b945e8/users/5d9b0e95b2e841739be689bcbe6d36a8/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 14:59:18.308130 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a695be8-1e07-4cf0-952a-11cb4727648b None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:18.308669 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a695be8-1e07-4cf0-952a-11cb4727648b None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:18.309108 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a695be8-1e07-4cf0-952a-11cb4727648b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:18.349104 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-3a695be8-1e07-4cf0-952a-11cb4727648b None None] MFA Rules not processed for user `5d9b0e95b2e841739be689bcbe6d36a8`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:18.350821 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-3a695be8-1e07-4cf0-952a-11cb4727648b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:18.460941 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 984/1950] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:18.463239 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810b44b0-b37c-4e09-a707-7d7f6b207d4b admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:18.562116 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810b44b0-b37c-4e09-a707-7d7f6b207d4b admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:18.578672 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4cc66-91ba-46af-8570-7436664ae27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:18.580163 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4cc66-91ba-46af-8570-7436664ae27b admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:18.580580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4cc66-91ba-46af-8570-7436664ae27b admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:18.580976 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4cc66-91ba-46af-8570-7436664ae27b admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:18.582696 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c4cc66-91ba-46af-8570-7436664ae27b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:18.586452 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c4cc66-91ba-46af-8570-7436664ae27b admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:18.638250 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a695be8-1e07-4cf0-952a-11cb4727648b None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:18.640244 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-3a695be8-1e07-4cf0-952a-11cb4727648b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:18.664926 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 967/1951] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:59:18 2022] POST /identity/v3/projects => generated 393 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:59:18.678261 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a695be8-1e07-4cf0-952a-11cb4727648b None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:18.680723 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4cc66-91ba-46af-8570-7436664ae27b admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:18.767675 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4cc66-91ba-46af-8570-7436664ae27b admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:18.772825 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa24379-312a-4e1c-bbbe-9278efaf6e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:18.773534 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa24379-312a-4e1c-bbbe-9278efaf6e13 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:18.774359 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa24379-312a-4e1c-bbbe-9278efaf6e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:18.774641 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa24379-312a-4e1c-bbbe-9278efaf6e13 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:18.777492 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa24379-312a-4e1c-bbbe-9278efaf6e13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:18.780220 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa24379-312a-4e1c-bbbe-9278efaf6e13 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:18.815831 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b665cf-ec6a-4eac-8ba7-5e0a3ba1ed9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:18.816044 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b665cf-ec6a-4eac-8ba7-5e0a3ba1ed9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:18.816270 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b665cf-ec6a-4eac-8ba7-5e0a3ba1ed9f admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:18.816638 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b665cf-ec6a-4eac-8ba7-5e0a3ba1ed9f admin admin] PATH_INFO: `/v3/projects/a26eb463d04e479a817558987b219290` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:18.818964 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b665cf-ec6a-4eac-8ba7-5e0a3ba1ed9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a26eb463d04e479a817558987b219290)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:18.821488 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b665cf-ec6a-4eac-8ba7-5e0a3ba1ed9f admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:18.828686 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 968/1952] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:59:18 2022] POST /identity/v3/users => generated 303 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:59:18.833670 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa24379-312a-4e1c-bbbe-9278efaf6e13 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:18.835917 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ffa24379-312a-4e1c-bbbe-9278efaf6e13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:18.850596 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa24379-312a-4e1c-bbbe-9278efaf6e13 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:18.857337 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40e98d1-fa22-41da-aacf-c684deba600c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:18.858413 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e98d1-fa22-41da-aacf-c684deba600c admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:18.858761 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e98d1-fa22-41da-aacf-c684deba600c admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:18.859028 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e98d1-fa22-41da-aacf-c684deba600c admin admin] PATH_INFO: `/v3/users/e1a39c411fc5431da160f2293d69b480` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:18.860266 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40e98d1-fa22-41da-aacf-c684deba600c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1a39c411fc5431da160f2293d69b480)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:18.862318 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40e98d1-fa22-41da-aacf-c684deba600c admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:18.887111 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-30b665cf-ec6a-4eac-8ba7-5e0a3ba1ed9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a26eb463d04e479a817558987b219290', 'request_id': 'req-30b665cf-ec6a-4eac-8ba7-5e0a3ba1ed9f'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:18.887312 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-30b665cf-ec6a-4eac-8ba7-5e0a3ba1ed9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a26eb463d04e479a817558987b219290', 'request_id': 'req-30b665cf-ec6a-4eac-8ba7-5e0a3ba1ed9f'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:18.896048 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-30b665cf-ec6a-4eac-8ba7-5e0a3ba1ed9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a26eb463d04e479a817558987b219290', 'request_id': 'req-30b665cf-ec6a-4eac-8ba7-5e0a3ba1ed9f'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:18.910260 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-30b665cf-ec6a-4eac-8ba7-5e0a3ba1ed9f admin admin] The token cache is being invalidate because project a26eb463d04e479a817558987b219290 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 14:59:18.910551 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-30b665cf-ec6a-4eac-8ba7-5e0a3ba1ed9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:18.911200 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-30b665cf-ec6a-4eac-8ba7-5e0a3ba1ed9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:18.912780 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 985/1953] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 14:59:18 2022] DELETE /identity/v3/projects/a26eb463d04e479a817558987b219290 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:18.913553 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b665cf-ec6a-4eac-8ba7-5e0a3ba1ed9f admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:18.915373 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-30b665cf-ec6a-4eac-8ba7-5e0a3ba1ed9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:18.935514 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e40e98d1-fa22-41da-aacf-c684deba600c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1a39c411fc5431da160f2293d69b480'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:18.936184 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e40e98d1-fa22-41da-aacf-c684deba600c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1a39c411fc5431da160f2293d69b480'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:18.943826 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e40e98d1-fa22-41da-aacf-c684deba600c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1a39c411fc5431da160f2293d69b480'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:18.951063 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e40e98d1-fa22-41da-aacf-c684deba600c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1a39c411fc5431da160f2293d69b480'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:18.951827 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e40e98d1-fa22-41da-aacf-c684deba600c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1a39c411fc5431da160f2293d69b480'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:18.973963 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 969/1954] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 14:59:18 2022] DELETE /identity/v3/users/e1a39c411fc5431da160f2293d69b480 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:18.976759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27134a16-a56c-4dd8-8610-8960aab15766 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:18.977719 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27134a16-a56c-4dd8-8610-8960aab15766 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:18.978060 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27134a16-a56c-4dd8-8610-8960aab15766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:18.981261 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b665cf-ec6a-4eac-8ba7-5e0a3ba1ed9f admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:18.996148 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-27134a16-a56c-4dd8-8610-8960aab15766 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:18.997386 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-27134a16-a56c-4dd8-8610-8960aab15766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:19.037183 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d781d8d-632e-4cc1-ac56-1428dca6cd18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:19.037874 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d781d8d-632e-4cc1-ac56-1428dca6cd18 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:19.038157 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d781d8d-632e-4cc1-ac56-1428dca6cd18 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:19.038430 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d781d8d-632e-4cc1-ac56-1428dca6cd18 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:19.039175 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d781d8d-632e-4cc1-ac56-1428dca6cd18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:19.041490 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d781d8d-632e-4cc1-ac56-1428dca6cd18 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:19.051664 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 986/1955] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:59:19.052987 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8b71f-d71c-4c76-b2c0-d3c80216c4e0 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:19.053281 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8b71f-d71c-4c76-b2c0-d3c80216c4e0 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:19.053563 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8b71f-d71c-4c76-b2c0-d3c80216c4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:19.075605 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-b6c8b71f-d71c-4c76-b2c0-d3c80216c4e0 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:19.076970 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-b6c8b71f-d71c-4c76-b2c0-d3c80216c4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:19.085355 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 987/1956] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:19.087004 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8b71f-d71c-4c76-b2c0-d3c80216c4e0 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:19.087635 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 970/1957] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:19.090464 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-b6c8b71f-d71c-4c76-b2c0-d3c80216c4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:19.090611 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427cc6b-b564-4c96-903b-2486a40a2df3 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:19.090798 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427cc6b-b564-4c96-903b-2486a40a2df3 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:19.091084 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427cc6b-b564-4c96-903b-2486a40a2df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:19.109643 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8b71f-d71c-4c76-b2c0-d3c80216c4e0 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:19.118259 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70060e-6c75-493d-ade1-7506044894f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:19.119195 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70060e-6c75-493d-ade1-7506044894f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:19.119644 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70060e-6c75-493d-ade1-7506044894f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:19.120341 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70060e-6c75-493d-ade1-7506044894f7 admin admin] PATH_INFO: `/v3/projects/cc3f1bc686034eef92c670510f400b64/users/fb7fbea7e0e34f01acd7482ebe4a76b0/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:19.125337 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-d427cc6b-b564-4c96-903b-2486a40a2df3 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:19.127392 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d427cc6b-b564-4c96-903b-2486a40a2df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:19.137926 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef70060e-6c75-493d-ade1-7506044894f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc3f1bc686034eef92c670510f400b64, user_id=fb7fbea7e0e34f01acd7482ebe4a76b0, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:19.139512 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 971/1958] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 14:59:19.140587 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d427cc6b-b564-4c96-903b-2486a40a2df3 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:19.141403 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef70060e-6c75-493d-ade1-7506044894f7 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:19.143340 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d427cc6b-b564-4c96-903b-2486a40a2df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:19.163623 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 988/1959] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:59:19 2022] PUT /identity/v3/projects/cc3f1bc686034eef92c670510f400b64/users/fb7fbea7e0e34f01acd7482ebe4a76b0/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:19.163623 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70060e-6c75-493d-ade1-7506044894f7 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:19.220222 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d427cc6b-b564-4c96-903b-2486a40a2df3 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:19.224150 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70060e-6c75-493d-ade1-7506044894f7 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:19.229108 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5deb78-9339-4686-8715-a5c0836c318e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:19.229775 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5deb78-9339-4686-8715-a5c0836c318e admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:19.230063 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5deb78-9339-4686-8715-a5c0836c318e admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:19.230345 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5deb78-9339-4686-8715-a5c0836c318e admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:19.231072 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5deb78-9339-4686-8715-a5c0836c318e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:59:19.231384 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5deb78-9339-4686-8715-a5c0836c318e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:19.233065 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5deb78-9339-4686-8715-a5c0836c318e admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:19.245614 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 989/1960] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 14:59:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:59:19.246441 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5deb78-9339-4686-8715-a5c0836c318e admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:19.255682 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5deb78-9339-4686-8715-a5c0836c318e admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:19.261771 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320bc2e4-3dba-4d17-ae39-99c6ef5ede2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:19.262634 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320bc2e4-3dba-4d17-ae39-99c6ef5ede2a admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:19.262923 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320bc2e4-3dba-4d17-ae39-99c6ef5ede2a admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:19.263203 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320bc2e4-3dba-4d17-ae39-99c6ef5ede2a admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:19.263928 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320bc2e4-3dba-4d17-ae39-99c6ef5ede2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:19.265557 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320bc2e4-3dba-4d17-ae39-99c6ef5ede2a admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:19.269070 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dd963b-8e67-42dc-8724-2c42ab07c02d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:19.269725 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd963b-8e67-42dc-8724-2c42ab07c02d service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:19.270005 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd963b-8e67-42dc-8724-2c42ab07c02d service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:19.270280 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd963b-8e67-42dc-8724-2c42ab07c02d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:19.282494 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 990/1961] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:59:19.282494 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320bc2e4-3dba-4d17-ae39-99c6ef5ede2a admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:19.285043 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dd963b-8e67-42dc-8724-2c42ab07c02d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:19.286497 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dd963b-8e67-42dc-8724-2c42ab07c02d service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:19.293078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320bc2e4-3dba-4d17-ae39-99c6ef5ede2a admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:19.298814 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 972/1962] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:59:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:59:19.302562 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f4057-6263-40da-90de-b5aa6076769d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:19.303319 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f4057-6263-40da-90de-b5aa6076769d admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:19.303644 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f4057-6263-40da-90de-b5aa6076769d admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:19.303952 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f4057-6263-40da-90de-b5aa6076769d admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:19.304759 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792f4057-6263-40da-90de-b5aa6076769d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:19.307046 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792f4057-6263-40da-90de-b5aa6076769d admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:19.307189 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dd963b-8e67-42dc-8724-2c42ab07c02d service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:19.319610 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dd963b-8e67-42dc-8724-2c42ab07c02d service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:19.326566 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce64d93-d057-476f-89a8-c3cd49326044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:19.327590 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce64d93-d057-476f-89a8-c3cd49326044 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:19.327976 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce64d93-d057-476f-89a8-c3cd49326044 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:19.328314 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce64d93-d057-476f-89a8-c3cd49326044 admin admin] PATH_INFO: `/v3/projects/cc3f1bc686034eef92c670510f400b64/users/fb7fbea7e0e34f01acd7482ebe4a76b0/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:19.332682 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce64d93-d057-476f-89a8-c3cd49326044 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc3f1bc686034eef92c670510f400b64, user_id=fb7fbea7e0e34f01acd7482ebe4a76b0, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:19.335457 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce64d93-d057-476f-89a8-c3cd49326044 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:19.340331 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 991/1963] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:59:19 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:59:19.344543 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f4057-6263-40da-90de-b5aa6076769d admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:19.350659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 973/1964] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:59:19 2022] PUT /identity/v3/projects/cc3f1bc686034eef92c670510f400b64/users/fb7fbea7e0e34f01acd7482ebe4a76b0/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:19.362320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5946c-1df7-4456-adfb-a1e54a8e2c38 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:19.362611 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5946c-1df7-4456-adfb-a1e54a8e2c38 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:19.362896 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5946c-1df7-4456-adfb-a1e54a8e2c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:19.392655 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-08b5946c-1df7-4456-adfb-a1e54a8e2c38 None None] MFA Rules not processed for user `fb7fbea7e0e34f01acd7482ebe4a76b0`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:19.394182 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-08b5946c-1df7-4456-adfb-a1e54a8e2c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:19.436545 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f4057-6263-40da-90de-b5aa6076769d admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:19.491762 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 974/1965] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:19 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 14:59:19.504358 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5946c-1df7-4456-adfb-a1e54a8e2c38 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:19.505903 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-08b5946c-1df7-4456-adfb-a1e54a8e2c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:19.516492 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa544b-0b39-4f3c-9d3b-8c6305776dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:19.517459 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa544b-0b39-4f3c-9d3b-8c6305776dc0 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:19.517768 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa544b-0b39-4f3c-9d3b-8c6305776dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:19.518211 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa544b-0b39-4f3c-9d3b-8c6305776dc0 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:19.519286 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa544b-0b39-4f3c-9d3b-8c6305776dc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:19.521656 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa544b-0b39-4f3c-9d3b-8c6305776dc0 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:19.552775 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 992/1966] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:59:19 2022] POST /identity/v3/users => generated 292 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:59:19.556245 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa544b-0b39-4f3c-9d3b-8c6305776dc0 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:19.568291 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa544b-0b39-4f3c-9d3b-8c6305776dc0 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:19.573516 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322292f-6c51-4f66-9150-ec5069c7051b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:19.574189 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322292f-6c51-4f66-9150-ec5069c7051b admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:19.574518 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322292f-6c51-4f66-9150-ec5069c7051b admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:19.574821 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322292f-6c51-4f66-9150-ec5069c7051b admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:19.575537 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e322292f-6c51-4f66-9150-ec5069c7051b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:19.577195 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e322292f-6c51-4f66-9150-ec5069c7051b admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:19.579565 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5946c-1df7-4456-adfb-a1e54a8e2c38 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:19.583582 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3fb770-9718-4091-90bd-c452ef4064c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:19.584294 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3fb770-9718-4091-90bd-c452ef4064c3 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:19.584581 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3fb770-9718-4091-90bd-c452ef4064c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:19.584902 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3fb770-9718-4091-90bd-c452ef4064c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:19.592779 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 993/1967] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:59:19.594527 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322292f-6c51-4f66-9150-ec5069c7051b admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:19.596250 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3fb770-9718-4091-90bd-c452ef4064c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:19.597821 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3fb770-9718-4091-90bd-c452ef4064c3 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:19.605498 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322292f-6c51-4f66-9150-ec5069c7051b admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:19.610515 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52493d-478f-4b4f-ade4-692ba829e762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:19.611232 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52493d-478f-4b4f-ade4-692ba829e762 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:19.611489 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52493d-478f-4b4f-ade4-692ba829e762 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:19.611785 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52493d-478f-4b4f-ade4-692ba829e762 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:19.612529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d52493d-478f-4b4f-ade4-692ba829e762 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:19.613159 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 975/1968] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:59:19 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 14:59:19.614234 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d52493d-478f-4b4f-ade4-692ba829e762 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:19.633386 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3fb770-9718-4091-90bd-c452ef4064c3 service cinder] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:19.635258 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 994/1969] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:59:19.635689 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-0c3fb770-9718-4091-90bd-c452ef4064c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:19.641052 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52493d-478f-4b4f-ade4-692ba829e762 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:19.656178 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52493d-478f-4b4f-ade4-692ba829e762 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:19.661582 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faacbf3-aaa9-4917-9a61-b1533d71e514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:19.662494 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3fb770-9718-4091-90bd-c452ef4064c3 service cinder] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:19.663497 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faacbf3-aaa9-4917-9a61-b1533d71e514 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:19.664119 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faacbf3-aaa9-4917-9a61-b1533d71e514 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:19.664566 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faacbf3-aaa9-4917-9a61-b1533d71e514 admin admin] PATH_INFO: `/v3/projects/873c823b233d47759b8bffddc49335a0/users/a7ce59406c5f4ffcb6eafeaeed88b956/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:19.675171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a06e5-de3f-4e1b-8a21-29a68d4d0f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:19.676306 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a06e5-de3f-4e1b-8a21-29a68d4d0f33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:19.676946 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a06e5-de3f-4e1b-8a21-29a68d4d0f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:19.678260 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a06e5-de3f-4e1b-8a21-29a68d4d0f33 admin admin] PATH_INFO: `/v3/projects/f794f875ef854455b4bb9c5e2e1bd59f` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:19.682730 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4a06e5-de3f-4e1b-8a21-29a68d4d0f33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f794f875ef854455b4bb9c5e2e1bd59f)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:19.686060 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faacbf3-aaa9-4917-9a61-b1533d71e514 admin admin] RBAC: Authorizing `identity:create_grant(project_id=873c823b233d47759b8bffddc49335a0, user_id=a7ce59406c5f4ffcb6eafeaeed88b956, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:19.688818 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4a06e5-de3f-4e1b-8a21-29a68d4d0f33 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:19.690118 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faacbf3-aaa9-4917-9a61-b1533d71e514 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:19.745747 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 995/1970] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:59:19 2022] PUT /identity/v3/projects/873c823b233d47759b8bffddc49335a0/users/a7ce59406c5f4ffcb6eafeaeed88b956/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:19.745747 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faacbf3-aaa9-4917-9a61-b1533d71e514 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:19.749251 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-0faacbf3-aaa9-4917-9a61-b1533d71e514 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:19.789838 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-cd4a06e5-de3f-4e1b-8a21-29a68d4d0f33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f794f875ef854455b4bb9c5e2e1bd59f', 'request_id': 'req-cd4a06e5-de3f-4e1b-8a21-29a68d4d0f33'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:19.799384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-cd4a06e5-de3f-4e1b-8a21-29a68d4d0f33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f794f875ef854455b4bb9c5e2e1bd59f', 'request_id': 'req-cd4a06e5-de3f-4e1b-8a21-29a68d4d0f33'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:19.813250 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-cd4a06e5-de3f-4e1b-8a21-29a68d4d0f33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f794f875ef854455b4bb9c5e2e1bd59f', 'request_id': 'req-cd4a06e5-de3f-4e1b-8a21-29a68d4d0f33'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:19.830273 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-cd4a06e5-de3f-4e1b-8a21-29a68d4d0f33 admin admin] The token cache is being invalidate because project f794f875ef854455b4bb9c5e2e1bd59f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 14:59:19.830900 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-cd4a06e5-de3f-4e1b-8a21-29a68d4d0f33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:19.831254 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-cd4a06e5-de3f-4e1b-8a21-29a68d4d0f33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:19.833346 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 976/1971] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 14:59:19 2022] DELETE /identity/v3/projects/f794f875ef854455b4bb9c5e2e1bd59f => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 14:59:19.834827 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a9a3d-c5ec-40ae-b858-5f5f0d608cfa None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:19.835110 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a9a3d-c5ec-40ae-b858-5f5f0d608cfa None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:19.835385 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a9a3d-c5ec-40ae-b858-5f5f0d608cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:19.845421 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faacbf3-aaa9-4917-9a61-b1533d71e514 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:19.866758 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-070a9a3d-c5ec-40ae-b858-5f5f0d608cfa None None] MFA Rules not processed for user `a7ce59406c5f4ffcb6eafeaeed88b956`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:19.868675 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-070a9a3d-c5ec-40ae-b858-5f5f0d608cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:19.926711 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d5ff8-8aa7-4b44-90f6-e1137efd9bb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:19.927403 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d5ff8-8aa7-4b44-90f6-e1137efd9bb8 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:19.927688 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d5ff8-8aa7-4b44-90f6-e1137efd9bb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:19.927960 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d5ff8-8aa7-4b44-90f6-e1137efd9bb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:19.933136 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-b16d5ff8-8aa7-4b44-90f6-e1137efd9bb8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:19.975173 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 977/1972] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:19.976180 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a9a3d-c5ec-40ae-b858-5f5f0d608cfa None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:19.979590 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-070a9a3d-c5ec-40ae-b858-5f5f0d608cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:20.021269 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16d5ff8-8aa7-4b44-90f6-e1137efd9bb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:20.023395 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16d5ff8-8aa7-4b44-90f6-e1137efd9bb8 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:20.035778 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 996/1973] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:59:19 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:59:20.036689 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d5ff8-8aa7-4b44-90f6-e1137efd9bb8 service cinder] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:20.038061 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-b16d5ff8-8aa7-4b44-90f6-e1137efd9bb8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:20.050614 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d5ff8-8aa7-4b44-90f6-e1137efd9bb8 service cinder] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:20.061434 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee645529-c8ff-4c94-a72c-a84c166b32af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:20.062320 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee645529-c8ff-4c94-a72c-a84c166b32af service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:20.062777 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee645529-c8ff-4c94-a72c-a84c166b32af service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:20.063197 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee645529-c8ff-4c94-a72c-a84c166b32af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:20.089692 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee645529-c8ff-4c94-a72c-a84c166b32af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:20.091507 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee645529-c8ff-4c94-a72c-a84c166b32af service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:20.097789 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a9a3d-c5ec-40ae-b858-5f5f0d608cfa None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:20.102414 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 997/1974] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:59:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:59:20.106352 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb66096-f529-4cab-bb65-854955b78269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:20.107722 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb66096-f529-4cab-bb65-854955b78269 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:20.108148 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb66096-f529-4cab-bb65-854955b78269 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:20.108572 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb66096-f529-4cab-bb65-854955b78269 admin admin] PATH_INFO: `/v3/users/207bbaf61a804d178ce156dc7600e6b9` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:20.111284 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb66096-f529-4cab-bb65-854955b78269 admin admin] RBAC: Authorizing `identity:delete_user(user_id=207bbaf61a804d178ce156dc7600e6b9)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:20.113340 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb66096-f529-4cab-bb65-854955b78269 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:20.127958 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee645529-c8ff-4c94-a72c-a84c166b32af service cinder] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:20.130136 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-ee645529-c8ff-4c94-a72c-a84c166b32af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:20.159595 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee645529-c8ff-4c94-a72c-a84c166b32af service cinder] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:20.166423 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453c09a-5c3b-43f2-b5a7-395d88b2f318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:20.168116 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453c09a-5c3b-43f2-b5a7-395d88b2f318 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:20.168471 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453c09a-5c3b-43f2-b5a7-395d88b2f318 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:20.168879 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453c09a-5c3b-43f2-b5a7-395d88b2f318 admin admin] PATH_INFO: `/v3/users/a7ce59406c5f4ffcb6eafeaeed88b956` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:20.170272 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a453c09a-5c3b-43f2-b5a7-395d88b2f318 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7ce59406c5f4ffcb6eafeaeed88b956)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:20.172645 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a453c09a-5c3b-43f2-b5a7-395d88b2f318 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:20.187338 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-3bb66096-f529-4cab-bb65-854955b78269 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '207bbaf61a804d178ce156dc7600e6b9'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:20.187995 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-3bb66096-f529-4cab-bb65-854955b78269 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '207bbaf61a804d178ce156dc7600e6b9'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:20.196485 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-3bb66096-f529-4cab-bb65-854955b78269 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '207bbaf61a804d178ce156dc7600e6b9'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:20.205364 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-3bb66096-f529-4cab-bb65-854955b78269 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '207bbaf61a804d178ce156dc7600e6b9'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:20.206332 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-3bb66096-f529-4cab-bb65-854955b78269 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '207bbaf61a804d178ce156dc7600e6b9'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:20.225477 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 978/1975] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 14:59:19 2022] DELETE /identity/v3/users/207bbaf61a804d178ce156dc7600e6b9 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:20.235298 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a453c09a-5c3b-43f2-b5a7-395d88b2f318 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7ce59406c5f4ffcb6eafeaeed88b956'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:20.244985 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a453c09a-5c3b-43f2-b5a7-395d88b2f318 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7ce59406c5f4ffcb6eafeaeed88b956'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:20.252573 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a453c09a-5c3b-43f2-b5a7-395d88b2f318 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7ce59406c5f4ffcb6eafeaeed88b956'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:20.253652 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a453c09a-5c3b-43f2-b5a7-395d88b2f318 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7ce59406c5f4ffcb6eafeaeed88b956'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:20.254368 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a453c09a-5c3b-43f2-b5a7-395d88b2f318 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7ce59406c5f4ffcb6eafeaeed88b956'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:20.276496 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 998/1976] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 14:59:20 2022] DELETE /identity/v3/users/a7ce59406c5f4ffcb6eafeaeed88b956 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 14:59:20.281917 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104fe03-cfba-4200-860d-a1db1c3df4c2 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:20.282309 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104fe03-cfba-4200-860d-a1db1c3df4c2 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:20.282732 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104fe03-cfba-4200-860d-a1db1c3df4c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:20.302281 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-e104fe03-cfba-4200-860d-a1db1c3df4c2 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:20.303677 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-e104fe03-cfba-4200-860d-a1db1c3df4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:20.407387 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 979/1977] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:20.414818 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453c09a-5c3b-43f2-b5a7-395d88b2f318 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:20.416309 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-a453c09a-5c3b-43f2-b5a7-395d88b2f318 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:20.479742 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453c09a-5c3b-43f2-b5a7-395d88b2f318 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:20.485202 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa0096f-11e5-48e7-aefa-9bfa25af110e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:20.485885 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0096f-11e5-48e7-aefa-9bfa25af110e service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:20.486173 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0096f-11e5-48e7-aefa-9bfa25af110e service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:20.486602 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0096f-11e5-48e7-aefa-9bfa25af110e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:20.498118 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa0096f-11e5-48e7-aefa-9bfa25af110e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:20.499802 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa0096f-11e5-48e7-aefa-9bfa25af110e service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:20.518402 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 999/1978] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:59:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:59:20.559268 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e104fe03-cfba-4200-860d-a1db1c3df4c2 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:20.560809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-e104fe03-cfba-4200-860d-a1db1c3df4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:20.584460 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e104fe03-cfba-4200-860d-a1db1c3df4c2 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:20.591193 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b81163-05ac-438e-817d-f5ef72716e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:20.592447 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b81163-05ac-438e-817d-f5ef72716e63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:20.593031 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b81163-05ac-438e-817d-f5ef72716e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:20.593475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b81163-05ac-438e-817d-f5ef72716e63 admin admin] PATH_INFO: `/v3/projects/ad416dc92ed1493caa1132d606b7cbd1` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:20.594848 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b81163-05ac-438e-817d-f5ef72716e63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad416dc92ed1493caa1132d606b7cbd1)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:20.597152 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b81163-05ac-438e-817d-f5ef72716e63 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:20.654924 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-b6b81163-05ac-438e-817d-f5ef72716e63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad416dc92ed1493caa1132d606b7cbd1', 'request_id': 'req-b6b81163-05ac-438e-817d-f5ef72716e63'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:20.661286 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-b6b81163-05ac-438e-817d-f5ef72716e63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad416dc92ed1493caa1132d606b7cbd1', 'request_id': 'req-b6b81163-05ac-438e-817d-f5ef72716e63'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:20.670015 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-b6b81163-05ac-438e-817d-f5ef72716e63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad416dc92ed1493caa1132d606b7cbd1', 'request_id': 'req-b6b81163-05ac-438e-817d-f5ef72716e63'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:20.682602 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-b6b81163-05ac-438e-817d-f5ef72716e63 admin admin] The token cache is being invalidate because project ad416dc92ed1493caa1132d606b7cbd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 14:59:20.682784 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-b6b81163-05ac-438e-817d-f5ef72716e63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:20.684660 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-b6b81163-05ac-438e-817d-f5ef72716e63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:20.691285 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 980/1979] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 14:59:20 2022] DELETE /identity/v3/projects/ad416dc92ed1493caa1132d606b7cbd1 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:20.693370 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566aa39f-30b9-4160-ab70-9d628bded5a2 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:20.693795 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566aa39f-30b9-4160-ab70-9d628bded5a2 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:20.694049 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566aa39f-30b9-4160-ab70-9d628bded5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:20.718955 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-566aa39f-30b9-4160-ab70-9d628bded5a2 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:20.721109 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-566aa39f-30b9-4160-ab70-9d628bded5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:20.789965 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1000/1980] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:20.798091 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0558e76-373f-4005-a24e-7d752e21d012 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:20.798386 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0558e76-373f-4005-a24e-7d752e21d012 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:20.798712 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0558e76-373f-4005-a24e-7d752e21d012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:20.818724 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-a0558e76-373f-4005-a24e-7d752e21d012 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:20.819929 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-a0558e76-373f-4005-a24e-7d752e21d012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:20.822383 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566aa39f-30b9-4160-ab70-9d628bded5a2 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:20.823942 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-566aa39f-30b9-4160-ab70-9d628bded5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:20.829730 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 981/1981] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:20.834384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0558e76-373f-4005-a24e-7d752e21d012 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:20.844408 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566aa39f-30b9-4160-ab70-9d628bded5a2 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:20.844781 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0558e76-373f-4005-a24e-7d752e21d012 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:20.849665 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5a2b92-b6db-448b-ae3a-ddbf67d905a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:20.850372 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a2b92-b6db-448b-ae3a-ddbf67d905a9 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:20.850579 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e716a09-911c-4185-855e-ea53b2452d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:20.850677 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a2b92-b6db-448b-ae3a-ddbf67d905a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:20.850995 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a2b92-b6db-448b-ae3a-ddbf67d905a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:20.852049 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e716a09-911c-4185-855e-ea53b2452d60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:20.852395 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e716a09-911c-4185-855e-ea53b2452d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:20.852523 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5a2b92-b6db-448b-ae3a-ddbf67d905a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:59:20.852708 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e716a09-911c-4185-855e-ea53b2452d60 admin admin] PATH_INFO: `/v3/projects/873c823b233d47759b8bffddc49335a0` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:20.853260 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5a2b92-b6db-448b-ae3a-ddbf67d905a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:20.854525 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e716a09-911c-4185-855e-ea53b2452d60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=873c823b233d47759b8bffddc49335a0)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:20.855053 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5a2b92-b6db-448b-ae3a-ddbf67d905a9 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:20.856711 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e716a09-911c-4185-855e-ea53b2452d60 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:20.872685 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 982/1982] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 14:59:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:59:20.877764 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5a2b92-b6db-448b-ae3a-ddbf67d905a9 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:20.894002 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5a2b92-b6db-448b-ae3a-ddbf67d905a9 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:20.902221 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a64c7a-a9bc-4625-8675-f11e94eb544d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:20.902964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a64c7a-a9bc-4625-8675-f11e94eb544d admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:20.903266 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a64c7a-a9bc-4625-8675-f11e94eb544d admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:20.903633 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a64c7a-a9bc-4625-8675-f11e94eb544d admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:20.904326 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a64c7a-a9bc-4625-8675-f11e94eb544d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:20.906711 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a64c7a-a9bc-4625-8675-f11e94eb544d admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:20.928398 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-4e716a09-911c-4185-855e-ea53b2452d60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '873c823b233d47759b8bffddc49335a0', 'request_id': 'req-4e716a09-911c-4185-855e-ea53b2452d60'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:20.938405 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-4e716a09-911c-4185-855e-ea53b2452d60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '873c823b233d47759b8bffddc49335a0', 'request_id': 'req-4e716a09-911c-4185-855e-ea53b2452d60'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:20.948423 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 983/1983] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:59:20 2022] POST /identity/v3/projects => generated 379 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:59:20.952727 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a64c7a-a9bc-4625-8675-f11e94eb544d admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:20.956310 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-4e716a09-911c-4185-855e-ea53b2452d60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '873c823b233d47759b8bffddc49335a0', 'request_id': 'req-4e716a09-911c-4185-855e-ea53b2452d60'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:20.973795 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-4e716a09-911c-4185-855e-ea53b2452d60 admin admin] The token cache is being invalidate because project 873c823b233d47759b8bffddc49335a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 14:59:20.974085 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-4e716a09-911c-4185-855e-ea53b2452d60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:20.975113 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-4e716a09-911c-4185-855e-ea53b2452d60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:20.976352 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1001/1984] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 14:59:20 2022] DELETE /identity/v3/projects/873c823b233d47759b8bffddc49335a0 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 14:59:20.982188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207ef3d-3e0b-4e63-9879-c79db731f512 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:20.982701 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207ef3d-3e0b-4e63-9879-c79db731f512 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:20.985949 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207ef3d-3e0b-4e63-9879-c79db731f512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:21.010357 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-9207ef3d-3e0b-4e63-9879-c79db731f512 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:21.011741 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-9207ef3d-3e0b-4e63-9879-c79db731f512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:21.045801 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a64c7a-a9bc-4625-8675-f11e94eb544d admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:21.051612 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cde882a-0c64-4363-8775-deb112ee320f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:21.053006 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde882a-0c64-4363-8775-deb112ee320f admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:21.053297 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde882a-0c64-4363-8775-deb112ee320f admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:21.053581 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde882a-0c64-4363-8775-deb112ee320f admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:21.054578 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cde882a-0c64-4363-8775-deb112ee320f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:21.057330 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cde882a-0c64-4363-8775-deb112ee320f admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:21.089384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 984/1985] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:59:20 2022] POST /identity/v3/users => generated 290 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 14:59:21.093585 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cde882a-0c64-4363-8775-deb112ee320f admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:21.095393 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-0cde882a-0c64-4363-8775-deb112ee320f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:21.103028 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1002/1986] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:21.111332 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661d491-de3d-4de9-a40b-48e3fae0afda None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:21.111644 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661d491-de3d-4de9-a40b-48e3fae0afda None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:21.111923 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661d491-de3d-4de9-a40b-48e3fae0afda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:21.115175 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cde882a-0c64-4363-8775-deb112ee320f admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:21.121418 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40377e86-8a9d-4e7a-b81c-cef4ccc6a270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:21.122095 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40377e86-8a9d-4e7a-b81c-cef4ccc6a270 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:21.122385 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40377e86-8a9d-4e7a-b81c-cef4ccc6a270 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:21.122751 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40377e86-8a9d-4e7a-b81c-cef4ccc6a270 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:21.123523 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40377e86-8a9d-4e7a-b81c-cef4ccc6a270 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:21.125109 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40377e86-8a9d-4e7a-b81c-cef4ccc6a270 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:21.131907 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-4661d491-de3d-4de9-a40b-48e3fae0afda None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:21.133708 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-4661d491-de3d-4de9-a40b-48e3fae0afda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:21.139986 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 985/1987] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:59:21.142281 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40377e86-8a9d-4e7a-b81c-cef4ccc6a270 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:21.145742 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1003/1988] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 14:59:21.146531 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4661d491-de3d-4de9-a40b-48e3fae0afda None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:21.156904 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4661d491-de3d-4de9-a40b-48e3fae0afda None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:21.164648 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40377e86-8a9d-4e7a-b81c-cef4ccc6a270 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:21.169451 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a4ab92-4d05-4478-a0b5-4a83fbe500d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:21.169790 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc4fd6b-a800-4149-9453-a407f19b8960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:21.170126 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4ab92-4d05-4478-a0b5-4a83fbe500d8 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:21.170410 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4ab92-4d05-4478-a0b5-4a83fbe500d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:21.172644 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4ab92-4d05-4478-a0b5-4a83fbe500d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:21.172644 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a4ab92-4d05-4478-a0b5-4a83fbe500d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:21.173259 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a4ab92-4d05-4478-a0b5-4a83fbe500d8 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:21.173896 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc4fd6b-a800-4149-9453-a407f19b8960 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:21.174935 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc4fd6b-a800-4149-9453-a407f19b8960 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:21.176974 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc4fd6b-a800-4149-9453-a407f19b8960 admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:21.177889 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc4fd6b-a800-4149-9453-a407f19b8960 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:59:21.178366 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc4fd6b-a800-4149-9453-a407f19b8960 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:21.180695 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc4fd6b-a800-4149-9453-a407f19b8960 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:21.187903 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 986/1989] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 06 14:59:21.188962 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a4ab92-4d05-4478-a0b5-4a83fbe500d8 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:21.200444 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a4ab92-4d05-4478-a0b5-4a83fbe500d8 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:21.203604 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1004/1990] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 14:59:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:59:21.208156 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc4fd6b-a800-4149-9453-a407f19b8960 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:21.211843 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e65a3c-37dc-4fd4-8f79-5ecdfd76527b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:21.212797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e65a3c-37dc-4fd4-8f79-5ecdfd76527b admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:21.213090 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e65a3c-37dc-4fd4-8f79-5ecdfd76527b admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:21.213406 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e65a3c-37dc-4fd4-8f79-5ecdfd76527b admin admin] PATH_INFO: `/v3/projects/a61c9690307a422180537982425434de/users/2a5ede5e6fd444a4a2dc0fe953705b33/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:21.219437 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc4fd6b-a800-4149-9453-a407f19b8960 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:21.226218 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d15db-f880-4575-966f-17e41af2484f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:21.227121 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d15db-f880-4575-966f-17e41af2484f admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:21.227687 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d15db-f880-4575-966f-17e41af2484f admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:21.228224 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d15db-f880-4575-966f-17e41af2484f admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:21.231825 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e65a3c-37dc-4fd4-8f79-5ecdfd76527b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a61c9690307a422180537982425434de, user_id=2a5ede5e6fd444a4a2dc0fe953705b33, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:21.232996 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d15db-f880-4575-966f-17e41af2484f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:21.234746 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e65a3c-37dc-4fd4-8f79-5ecdfd76527b admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:21.235254 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d15db-f880-4575-966f-17e41af2484f admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:21.254298 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 987/1991] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:59:21 2022] PUT /identity/v3/projects/a61c9690307a422180537982425434de/users/2a5ede5e6fd444a4a2dc0fe953705b33/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 14:59:21.266084 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e70f23b-23c6-473c-8227-05a877c44b94 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:21.266379 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e70f23b-23c6-473c-8227-05a877c44b94 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:21.266662 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e70f23b-23c6-473c-8227-05a877c44b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:21.277890 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1005/1992] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:59:21 2022] POST /identity/v3/projects => generated 383 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:59:21.282091 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d15db-f880-4575-966f-17e41af2484f admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:21.306127 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-8e70f23b-23c6-473c-8227-05a877c44b94 None None] MFA Rules not processed for user `2a5ede5e6fd444a4a2dc0fe953705b33`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:21.307640 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-8e70f23b-23c6-473c-8227-05a877c44b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:21.355015 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d15db-f880-4575-966f-17e41af2484f admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:21.359767 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0244b-e82a-4cf8-8f84-bf4cc8e4041d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:21.360440 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0244b-e82a-4cf8-8f84-bf4cc8e4041d admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:21.360755 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0244b-e82a-4cf8-8f84-bf4cc8e4041d admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:21.361027 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0244b-e82a-4cf8-8f84-bf4cc8e4041d admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:21.361712 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e0244b-e82a-4cf8-8f84-bf4cc8e4041d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:21.363714 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e0244b-e82a-4cf8-8f84-bf4cc8e4041d admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:21.385102 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 988/1993] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 14:59:21.393271 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1006/1994] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:59:21 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 14:59:21.393394 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e70f23b-23c6-473c-8227-05a877c44b94 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:21.395449 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0244b-e82a-4cf8-8f84-bf4cc8e4041d admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:21.403542 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e70f23b-23c6-473c-8227-05a877c44b94 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:21.406839 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0244b-e82a-4cf8-8f84-bf4cc8e4041d admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:21.408555 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98779589-fdaa-4638-8e0c-ff82daf8b3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:21.409251 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98779589-fdaa-4638-8e0c-ff82daf8b3fc admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:21.409533 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98779589-fdaa-4638-8e0c-ff82daf8b3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:21.409813 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98779589-fdaa-4638-8e0c-ff82daf8b3fc admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:21.410582 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98779589-fdaa-4638-8e0c-ff82daf8b3fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:21.412472 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98779589-fdaa-4638-8e0c-ff82daf8b3fc admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:21.412921 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2455063-2018-43c2-b3f1-6592471a08a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:21.413588 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2455063-2018-43c2-b3f1-6592471a08a0 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:21.413870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2455063-2018-43c2-b3f1-6592471a08a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:21.414247 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2455063-2018-43c2-b3f1-6592471a08a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:21.415417 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2455063-2018-43c2-b3f1-6592471a08a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:21.417347 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2455063-2018-43c2-b3f1-6592471a08a0 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:21.432423 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1007/1995] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 06 14:59:21.434065 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2455063-2018-43c2-b3f1-6592471a08a0 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:21.440182 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 989/1996] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:59:21 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:59:21.444700 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98779589-fdaa-4638-8e0c-ff82daf8b3fc admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:21.451724 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2455063-2018-43c2-b3f1-6592471a08a0 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:21.514471 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98779589-fdaa-4638-8e0c-ff82daf8b3fc admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:21.520273 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f81752f-85a8-4c0c-81b4-42027416a0bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:21.521074 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81752f-85a8-4c0c-81b4-42027416a0bb admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:21.521351 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81752f-85a8-4c0c-81b4-42027416a0bb admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:21.521631 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81752f-85a8-4c0c-81b4-42027416a0bb admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:21.522016 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09a5f2-ffb2-4ebf-bb37-b689a94f9436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:21.522352 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f81752f-85a8-4c0c-81b4-42027416a0bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:21.522839 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09a5f2-ffb2-4ebf-bb37-b689a94f9436 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:21.523193 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09a5f2-ffb2-4ebf-bb37-b689a94f9436 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:21.523531 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09a5f2-ffb2-4ebf-bb37-b689a94f9436 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:21.524270 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f81752f-85a8-4c0c-81b4-42027416a0bb admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:21.524744 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb09a5f2-ffb2-4ebf-bb37-b689a94f9436 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:21.528974 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb09a5f2-ffb2-4ebf-bb37-b689a94f9436 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:21.534884 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1008/1997] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 06 14:59:21.539039 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f81752f-85a8-4c0c-81b4-42027416a0bb admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:21.549751 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f81752f-85a8-4c0c-81b4-42027416a0bb admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:21.560763 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9c661-4346-48f1-b9b4-6f37b36be6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:21.561794 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9c661-4346-48f1-b9b4-6f37b36be6eb admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:21.562153 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9c661-4346-48f1-b9b4-6f37b36be6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:21.562496 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9c661-4346-48f1-b9b4-6f37b36be6eb admin admin] PATH_INFO: `/v3/projects/170aa3ef89474632955b143b4665b6e6/users/19347c6703b74caebef5c713305945fd/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:21.567389 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 990/1998] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:59:21 2022] POST /identity/v3/users => generated 297 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 14:59:21.569104 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09a5f2-ffb2-4ebf-bb37-b689a94f9436 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:21.580725 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09a5f2-ffb2-4ebf-bb37-b689a94f9436 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:21.583645 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da9c661-4346-48f1-b9b4-6f37b36be6eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=170aa3ef89474632955b143b4665b6e6, user_id=19347c6703b74caebef5c713305945fd, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:21.585530 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240f936-1bc0-4854-9edd-26f2281638ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:21.586225 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240f936-1bc0-4854-9edd-26f2281638ff admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:21.586693 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240f936-1bc0-4854-9edd-26f2281638ff admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:21.587268 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da9c661-4346-48f1-b9b4-6f37b36be6eb admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:21.587383 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240f936-1bc0-4854-9edd-26f2281638ff admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:21.587874 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a240f936-1bc0-4854-9edd-26f2281638ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:21.589487 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a240f936-1bc0-4854-9edd-26f2281638ff admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:21.603787 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 991/1999] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:59:21.607656 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1009/2000] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:59:21 2022] PUT /identity/v3/projects/170aa3ef89474632955b143b4665b6e6/users/19347c6703b74caebef5c713305945fd/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 14:59:21.609404 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240f936-1bc0-4854-9edd-26f2281638ff admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:21.618733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36963b24-f25e-4343-b361-e187bee12ce1 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:21.619029 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36963b24-f25e-4343-b361-e187bee12ce1 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:21.619570 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36963b24-f25e-4343-b361-e187bee12ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:21.621922 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240f936-1bc0-4854-9edd-26f2281638ff admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:21.656886 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-36963b24-f25e-4343-b361-e187bee12ce1 None None] MFA Rules not processed for user `19347c6703b74caebef5c713305945fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:21.658220 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-36963b24-f25e-4343-b361-e187bee12ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:21.701931 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f725cb4-4c7a-417d-897f-a08a95596d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:21.702841 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f725cb4-4c7a-417d-897f-a08a95596d0a admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:21.703151 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f725cb4-4c7a-417d-897f-a08a95596d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:21.703446 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f725cb4-4c7a-417d-897f-a08a95596d0a admin admin] PATH_INFO: `/v3/projects/e551e1e3bc444dbc918d2d0e0fafe3bf/users/f0e17c73864d42ecbef543f9186095bf/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:21.718749 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f725cb4-4c7a-417d-897f-a08a95596d0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e551e1e3bc444dbc918d2d0e0fafe3bf, user_id=f0e17c73864d42ecbef543f9186095bf, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:21.721646 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f725cb4-4c7a-417d-897f-a08a95596d0a admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:21.741389 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1010/2001] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:21.744534 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 992/2002] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:59:21 2022] PUT /identity/v3/projects/e551e1e3bc444dbc918d2d0e0fafe3bf/users/f0e17c73864d42ecbef543f9186095bf/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:21.748512 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36963b24-f25e-4343-b361-e187bee12ce1 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:21.754321 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f725cb4-4c7a-417d-897f-a08a95596d0a admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:21.757204 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-9f725cb4-4c7a-417d-897f-a08a95596d0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:21.763389 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36963b24-f25e-4343-b361-e187bee12ce1 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:21.826393 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f725cb4-4c7a-417d-897f-a08a95596d0a admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:21.829816 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12789d6-602c-4268-bb7f-6d600e5960a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:21.830506 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12789d6-602c-4268-bb7f-6d600e5960a9 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:21.830796 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12789d6-602c-4268-bb7f-6d600e5960a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:21.831249 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12789d6-602c-4268-bb7f-6d600e5960a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:21.831396 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53578c5-4654-4b75-a856-83578c8508f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:21.831995 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12789d6-602c-4268-bb7f-6d600e5960a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:21.832267 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53578c5-4654-4b75-a856-83578c8508f8 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:21.832833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53578c5-4654-4b75-a856-83578c8508f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:21.833212 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53578c5-4654-4b75-a856-83578c8508f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:21.833650 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12789d6-602c-4268-bb7f-6d600e5960a9 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:21.844656 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1011/2003] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:59:21.848832 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12789d6-602c-4268-bb7f-6d600e5960a9 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:21.859365 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12789d6-602c-4268-bb7f-6d600e5960a9 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:21.868206 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8789cf-4867-464c-b873-d6dca0df6ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:21.869367 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8789cf-4867-464c-b873-d6dca0df6ae7 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:21.869610 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8789cf-4867-464c-b873-d6dca0df6ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:21.870459 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8789cf-4867-464c-b873-d6dca0df6ae7 admin admin] PATH_INFO: `/v3/projects/e551e1e3bc444dbc918d2d0e0fafe3bf/users/f0e17c73864d42ecbef543f9186095bf/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:21.873363 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8789cf-4867-464c-b873-d6dca0df6ae7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e551e1e3bc444dbc918d2d0e0fafe3bf, user_id=f0e17c73864d42ecbef543f9186095bf, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:21.876037 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8789cf-4867-464c-b873-d6dca0df6ae7 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:21.889947 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1012/2004] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:59:21 2022] PUT /identity/v3/projects/e551e1e3bc444dbc918d2d0e0fafe3bf/users/f0e17c73864d42ecbef543f9186095bf/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 14:59:21.901135 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b1187-4ec1-45c8-875e-78eadff398d8 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:21.901604 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b1187-4ec1-45c8-875e-78eadff398d8 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:21.902068 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b1187-4ec1-45c8-875e-78eadff398d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:21.912571 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53578c5-4654-4b75-a856-83578c8508f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:21.915947 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53578c5-4654-4b75-a856-83578c8508f8 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:21.937696 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 993/2005] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:59:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 14:59:21.940323 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-888b1187-4ec1-45c8-875e-78eadff398d8 None None] MFA Rules not processed for user `f0e17c73864d42ecbef543f9186095bf`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:21.941629 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-888b1187-4ec1-45c8-875e-78eadff398d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:22.029957 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1013/2006] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:21 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 14:59:22.104769 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53578c5-4654-4b75-a856-83578c8508f8 service cinder] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:22.107491 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-f53578c5-4654-4b75-a856-83578c8508f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:22.121399 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53578c5-4654-4b75-a856-83578c8508f8 service cinder] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:22.229998 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fba6ad-0089-4ff1-a8d0-7845e3deb50c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:22.231082 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fba6ad-0089-4ff1-a8d0-7845e3deb50c service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:22.231385 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fba6ad-0089-4ff1-a8d0-7845e3deb50c service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:22.231746 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fba6ad-0089-4ff1-a8d0-7845e3deb50c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:22.256166 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fba6ad-0089-4ff1-a8d0-7845e3deb50c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:22.257835 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fba6ad-0089-4ff1-a8d0-7845e3deb50c service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:22.292655 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 994/2007] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:59:22 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:59:22.931424 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b1187-4ec1-45c8-875e-78eadff398d8 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:22.995991 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b1187-4ec1-45c8-875e-78eadff398d8 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:23.001207 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3780ce94-7db4-4dd7-ab42-d2c030964dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:23.002109 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3780ce94-7db4-4dd7-ab42-d2c030964dbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:23.002448 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3780ce94-7db4-4dd7-ab42-d2c030964dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:23.002728 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3780ce94-7db4-4dd7-ab42-d2c030964dbd admin admin] PATH_INFO: `/v3/users/2a5ede5e6fd444a4a2dc0fe953705b33` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:23.004200 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3780ce94-7db4-4dd7-ab42-d2c030964dbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a5ede5e6fd444a4a2dc0fe953705b33)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:23.006191 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3780ce94-7db4-4dd7-ab42-d2c030964dbd admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:23.057116 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-3780ce94-7db4-4dd7-ab42-d2c030964dbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a5ede5e6fd444a4a2dc0fe953705b33'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:23.065035 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-3780ce94-7db4-4dd7-ab42-d2c030964dbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a5ede5e6fd444a4a2dc0fe953705b33'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:23.072742 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-3780ce94-7db4-4dd7-ab42-d2c030964dbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a5ede5e6fd444a4a2dc0fe953705b33'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:23.073414 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-3780ce94-7db4-4dd7-ab42-d2c030964dbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a5ede5e6fd444a4a2dc0fe953705b33'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:23.074037 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-3780ce94-7db4-4dd7-ab42-d2c030964dbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a5ede5e6fd444a4a2dc0fe953705b33'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:23.091137 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1014/2008] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 14:59:22 2022] DELETE /identity/v3/users/2a5ede5e6fd444a4a2dc0fe953705b33 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:23.095615 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7310810-dbd9-4008-ac74-fd213e3e7ae2 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:23.096111 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7310810-dbd9-4008-ac74-fd213e3e7ae2 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:23.096320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7310810-dbd9-4008-ac74-fd213e3e7ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:23.114965 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-a7310810-dbd9-4008-ac74-fd213e3e7ae2 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:23.116187 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-a7310810-dbd9-4008-ac74-fd213e3e7ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:23.204655 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 995/2009] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:23.217002 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3780ce94-7db4-4dd7-ab42-d2c030964dbd admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:23.218442 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-3780ce94-7db4-4dd7-ab42-d2c030964dbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:23.292156 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3780ce94-7db4-4dd7-ab42-d2c030964dbd admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:23.296479 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f80cae-a29d-4032-a156-4b5cda6688bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:23.297197 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f80cae-a29d-4032-a156-4b5cda6688bc service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:23.297479 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f80cae-a29d-4032-a156-4b5cda6688bc service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:23.297784 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f80cae-a29d-4032-a156-4b5cda6688bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:23.308067 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f80cae-a29d-4032-a156-4b5cda6688bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:23.309566 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f80cae-a29d-4032-a156-4b5cda6688bc service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:23.320975 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1015/2010] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:59:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:59:23.763398 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7310810-dbd9-4008-ac74-fd213e3e7ae2 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:23.766400 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-a7310810-dbd9-4008-ac74-fd213e3e7ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:23.779875 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7310810-dbd9-4008-ac74-fd213e3e7ae2 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:23.786068 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0554ad-dc9f-4bc6-8df9-ef238b32d794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:23.787100 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0554ad-dc9f-4bc6-8df9-ef238b32d794 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:23.787386 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0554ad-dc9f-4bc6-8df9-ef238b32d794 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:23.787663 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0554ad-dc9f-4bc6-8df9-ef238b32d794 admin admin] PATH_INFO: `/v3/projects/a61c9690307a422180537982425434de` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:23.789740 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0554ad-dc9f-4bc6-8df9-ef238b32d794 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a61c9690307a422180537982425434de)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:23.792180 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0554ad-dc9f-4bc6-8df9-ef238b32d794 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:23.858540 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-cf0554ad-dc9f-4bc6-8df9-ef238b32d794 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a61c9690307a422180537982425434de', 'request_id': 'req-cf0554ad-dc9f-4bc6-8df9-ef238b32d794'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:23.865088 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-cf0554ad-dc9f-4bc6-8df9-ef238b32d794 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a61c9690307a422180537982425434de', 'request_id': 'req-cf0554ad-dc9f-4bc6-8df9-ef238b32d794'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:23.876815 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-cf0554ad-dc9f-4bc6-8df9-ef238b32d794 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a61c9690307a422180537982425434de', 'request_id': 'req-cf0554ad-dc9f-4bc6-8df9-ef238b32d794'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:23.889819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-cf0554ad-dc9f-4bc6-8df9-ef238b32d794 admin admin] The token cache is being invalidate because project a61c9690307a422180537982425434de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 14:59:23.890264 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-cf0554ad-dc9f-4bc6-8df9-ef238b32d794 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:23.891130 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-cf0554ad-dc9f-4bc6-8df9-ef238b32d794 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:23.892888 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 996/2011] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 14:59:23 2022] DELETE /identity/v3/projects/a61c9690307a422180537982425434de => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:23.897702 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f80cae-a29d-4032-a156-4b5cda6688bc service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:23.909325 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f80cae-a29d-4032-a156-4b5cda6688bc service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:23.982044 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe58eca-9ede-44e3-bff2-662c4b48e804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:23.983820 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe58eca-9ede-44e3-bff2-662c4b48e804 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:23.984276 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe58eca-9ede-44e3-bff2-662c4b48e804 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:23.984732 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe58eca-9ede-44e3-bff2-662c4b48e804 admin admin] PATH_INFO: `/v3/users/f0e17c73864d42ecbef543f9186095bf` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:23.986390 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe58eca-9ede-44e3-bff2-662c4b48e804 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0e17c73864d42ecbef543f9186095bf)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:23.988566 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe58eca-9ede-44e3-bff2-662c4b48e804 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:24.000739 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0554ad-dc9f-4bc6-8df9-ef238b32d794 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:24.002186 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-cf0554ad-dc9f-4bc6-8df9-ef238b32d794 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:24.018376 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0554ad-dc9f-4bc6-8df9-ef238b32d794 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:24.026046 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e1f81-b280-4d70-b972-286626128b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:24.027029 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e1f81-b280-4d70-b972-286626128b41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:24.027369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e1f81-b280-4d70-b972-286626128b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:24.027770 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e1f81-b280-4d70-b972-286626128b41 admin admin] PATH_INFO: `/v3/users/5d9b0e95b2e841739be689bcbe6d36a8` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:24.029324 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2e1f81-b280-4d70-b972-286626128b41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d9b0e95b2e841739be689bcbe6d36a8)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:24.032115 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2e1f81-b280-4d70-b972-286626128b41 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:24.052513 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-dfe58eca-9ede-44e3-bff2-662c4b48e804 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0e17c73864d42ecbef543f9186095bf'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:24.063035 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-dfe58eca-9ede-44e3-bff2-662c4b48e804 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0e17c73864d42ecbef543f9186095bf'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:24.072778 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-dfe58eca-9ede-44e3-bff2-662c4b48e804 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0e17c73864d42ecbef543f9186095bf'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:24.073627 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-dfe58eca-9ede-44e3-bff2-662c4b48e804 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0e17c73864d42ecbef543f9186095bf'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:24.074323 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-dfe58eca-9ede-44e3-bff2-662c4b48e804 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0e17c73864d42ecbef543f9186095bf'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:24.085132 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-fa2e1f81-b280-4d70-b972-286626128b41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d9b0e95b2e841739be689bcbe6d36a8'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:24.085929 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-fa2e1f81-b280-4d70-b972-286626128b41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d9b0e95b2e841739be689bcbe6d36a8'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:24.093737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1016/2012] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 14:59:23 2022] DELETE /identity/v3/users/f0e17c73864d42ecbef543f9186095bf => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:24.098123 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-fa2e1f81-b280-4d70-b972-286626128b41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d9b0e95b2e841739be689bcbe6d36a8'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:24.109307 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-fa2e1f81-b280-4d70-b972-286626128b41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d9b0e95b2e841739be689bcbe6d36a8'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:24.110242 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-fa2e1f81-b280-4d70-b972-286626128b41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d9b0e95b2e841739be689bcbe6d36a8'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:24.136113 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 997/2013] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 14:59:23 2022] DELETE /identity/v3/users/5d9b0e95b2e841739be689bcbe6d36a8 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 14:59:24.141864 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f85329-281d-441a-8434-5b7c9e1f88db None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:24.142319 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f85329-281d-441a-8434-5b7c9e1f88db None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:24.142854 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f85329-281d-441a-8434-5b7c9e1f88db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:24.168469 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-74f85329-281d-441a-8434-5b7c9e1f88db None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:24.169697 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-74f85329-281d-441a-8434-5b7c9e1f88db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:24.243327 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1017/2014] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:24.255444 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e1f81-b280-4d70-b972-286626128b41 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:24.257088 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-fa2e1f81-b280-4d70-b972-286626128b41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:24.333595 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e1f81-b280-4d70-b972-286626128b41 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:24.340403 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e470b6d6-abf6-451d-99d7-868b0c757e6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:24.342231 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470b6d6-abf6-451d-99d7-868b0c757e6c service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:24.342667 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470b6d6-abf6-451d-99d7-868b0c757e6c service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:24.343322 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470b6d6-abf6-451d-99d7-868b0c757e6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:24.359533 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e470b6d6-abf6-451d-99d7-868b0c757e6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:24.361444 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e470b6d6-abf6-451d-99d7-868b0c757e6c service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:24.372848 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 998/2015] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:59:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 14:59:24.434901 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f85329-281d-441a-8434-5b7c9e1f88db None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:24.436565 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-74f85329-281d-441a-8434-5b7c9e1f88db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:24.451483 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f85329-281d-441a-8434-5b7c9e1f88db None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:24.457874 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2db919-c263-48d3-b039-29667fc6517d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:24.465667 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2db919-c263-48d3-b039-29667fc6517d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:24.465975 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2db919-c263-48d3-b039-29667fc6517d admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:24.466190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2db919-c263-48d3-b039-29667fc6517d admin admin] PATH_INFO: `/v3/projects/e551e1e3bc444dbc918d2d0e0fafe3bf` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:24.467517 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2db919-c263-48d3-b039-29667fc6517d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e551e1e3bc444dbc918d2d0e0fafe3bf)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:24.469495 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2db919-c263-48d3-b039-29667fc6517d admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:24.535103 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-dc2db919-c263-48d3-b039-29667fc6517d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e551e1e3bc444dbc918d2d0e0fafe3bf', 'request_id': 'req-dc2db919-c263-48d3-b039-29667fc6517d'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:24.543956 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-dc2db919-c263-48d3-b039-29667fc6517d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e551e1e3bc444dbc918d2d0e0fafe3bf', 'request_id': 'req-dc2db919-c263-48d3-b039-29667fc6517d'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:24.552766 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-dc2db919-c263-48d3-b039-29667fc6517d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e551e1e3bc444dbc918d2d0e0fafe3bf', 'request_id': 'req-dc2db919-c263-48d3-b039-29667fc6517d'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:24.564329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-dc2db919-c263-48d3-b039-29667fc6517d admin admin] The token cache is being invalidate because project e551e1e3bc444dbc918d2d0e0fafe3bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 14:59:24.564643 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-dc2db919-c263-48d3-b039-29667fc6517d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:24.565297 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-dc2db919-c263-48d3-b039-29667fc6517d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:24.566878 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1018/2016] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 14:59:24 2022] DELETE /identity/v3/projects/e551e1e3bc444dbc918d2d0e0fafe3bf => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:24.572850 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a2dd6-84de-4a83-af9e-6520f585e870 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:24.573245 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a2dd6-84de-4a83-af9e-6520f585e870 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:24.573626 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a2dd6-84de-4a83-af9e-6520f585e870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:24.594349 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-357a2dd6-84de-4a83-af9e-6520f585e870 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:24.595565 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-357a2dd6-84de-4a83-af9e-6520f585e870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:24.660208 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 999/2017] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:24.671634 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b644c-6176-482b-b0fe-be59f2371fc6 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:24.671941 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b644c-6176-482b-b0fe-be59f2371fc6 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:24.672274 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b644c-6176-482b-b0fe-be59f2371fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:24.702902 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-e12b644c-6176-482b-b0fe-be59f2371fc6 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:24.704829 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e12b644c-6176-482b-b0fe-be59f2371fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:24.739443 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1019/2018] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:24.747450 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a2dd6-84de-4a83-af9e-6520f585e870 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:24.760179 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a2dd6-84de-4a83-af9e-6520f585e870 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:24.765806 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f51f2-d39b-4ea9-b60d-6204e260aeba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:24.766499 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f51f2-d39b-4ea9-b60d-6204e260aeba admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:24.766788 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f51f2-d39b-4ea9-b60d-6204e260aeba admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:24.767070 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f51f2-d39b-4ea9-b60d-6204e260aeba admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:24.767818 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54f51f2-d39b-4ea9-b60d-6204e260aeba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:59:24.768127 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54f51f2-d39b-4ea9-b60d-6204e260aeba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:24.769834 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54f51f2-d39b-4ea9-b60d-6204e260aeba admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:24.794092 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b644c-6176-482b-b0fe-be59f2371fc6 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:24.797050 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1000/2019] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 14:59:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:59:24.806503 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b644c-6176-482b-b0fe-be59f2371fc6 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:24.812518 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e2a5c-8947-4a20-a93a-24f4850c566c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:24.813309 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e2a5c-8947-4a20-a93a-24f4850c566c admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:24.813668 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e2a5c-8947-4a20-a93a-24f4850c566c admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:24.814043 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e2a5c-8947-4a20-a93a-24f4850c566c admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:24.815455 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101e2a5c-8947-4a20-a93a-24f4850c566c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:24.818041 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101e2a5c-8947-4a20-a93a-24f4850c566c admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:24.866810 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1020/2020] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:59:24 2022] POST /identity/v3/projects => generated 377 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:59:24.872833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f51f2-d39b-4ea9-b60d-6204e260aeba admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:24.884665 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f51f2-d39b-4ea9-b60d-6204e260aeba admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:24.898963 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e2a5c-8947-4a20-a93a-24f4850c566c admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:24.901189 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-101e2a5c-8947-4a20-a93a-24f4850c566c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:24.980450 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f11217-83e8-43f3-9c6d-a4b508a36830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:24.981190 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f11217-83e8-43f3-9c6d-a4b508a36830 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:24.982228 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f11217-83e8-43f3-9c6d-a4b508a36830 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:24.983245 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f11217-83e8-43f3-9c6d-a4b508a36830 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:24.984109 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f11217-83e8-43f3-9c6d-a4b508a36830 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:24.986603 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f11217-83e8-43f3-9c6d-a4b508a36830 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:25.012775 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1001/2021] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:59:24 2022] POST /identity/v3/users => generated 289 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:59:25.016787 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f11217-83e8-43f3-9c6d-a4b508a36830 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:25.028191 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f11217-83e8-43f3-9c6d-a4b508a36830 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:25.032831 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e2a5c-8947-4a20-a93a-24f4850c566c admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:25.034402 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c29e9d-ef6c-470d-be87-2845c32eec07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:25.035320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c29e9d-ef6c-470d-be87-2845c32eec07 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:25.035757 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c29e9d-ef6c-470d-be87-2845c32eec07 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:25.036337 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c29e9d-ef6c-470d-be87-2845c32eec07 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:25.037260 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c29e9d-ef6c-470d-be87-2845c32eec07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:25.038450 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51ba342-6017-4177-8c56-c4091ebb98ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:25.039377 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c29e9d-ef6c-470d-be87-2845c32eec07 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:25.040473 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ba342-6017-4177-8c56-c4091ebb98ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:25.040801 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ba342-6017-4177-8c56-c4091ebb98ff admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:25.041091 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ba342-6017-4177-8c56-c4091ebb98ff admin admin] PATH_INFO: `/v3/projects/192b34a3662e4360b8e8450ad7b945e8` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:25.042898 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51ba342-6017-4177-8c56-c4091ebb98ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=192b34a3662e4360b8e8450ad7b945e8)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:25.045222 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51ba342-6017-4177-8c56-c4091ebb98ff admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:25.060929 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1002/2022] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 06 14:59:25.062177 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c29e9d-ef6c-470d-be87-2845c32eec07 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:25.074858 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c29e9d-ef6c-470d-be87-2845c32eec07 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:25.083653 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5cccb-c529-4ef6-ad79-389fbc26fb64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:25.085411 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5cccb-c529-4ef6-ad79-389fbc26fb64 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:25.086775 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5cccb-c529-4ef6-ad79-389fbc26fb64 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:25.087110 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5cccb-c529-4ef6-ad79-389fbc26fb64 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:25.088122 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb5cccb-c529-4ef6-ad79-389fbc26fb64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:25.090555 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb5cccb-c529-4ef6-ad79-389fbc26fb64 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:25.107698 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1003/2023] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:59:25.111858 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5cccb-c529-4ef6-ad79-389fbc26fb64 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:25.123798 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f51ba342-6017-4177-8c56-c4091ebb98ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '192b34a3662e4360b8e8450ad7b945e8', 'request_id': 'req-f51ba342-6017-4177-8c56-c4091ebb98ff'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:25.124894 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5cccb-c529-4ef6-ad79-389fbc26fb64 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:25.129949 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f51ba342-6017-4177-8c56-c4091ebb98ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '192b34a3662e4360b8e8450ad7b945e8', 'request_id': 'req-f51ba342-6017-4177-8c56-c4091ebb98ff'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:25.141570 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f51ba342-6017-4177-8c56-c4091ebb98ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '192b34a3662e4360b8e8450ad7b945e8', 'request_id': 'req-f51ba342-6017-4177-8c56-c4091ebb98ff'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:25.168422 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f51ba342-6017-4177-8c56-c4091ebb98ff admin admin] The token cache is being invalidate because project 192b34a3662e4360b8e8450ad7b945e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 14:59:25.169264 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f51ba342-6017-4177-8c56-c4091ebb98ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:25.170272 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f51ba342-6017-4177-8c56-c4091ebb98ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:25.172645 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1021/2024] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 14:59:24 2022] DELETE /identity/v3/projects/192b34a3662e4360b8e8450ad7b945e8 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 14:59:25.177236 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51ba342-6017-4177-8c56-c4091ebb98ff admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:25.178675 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f51ba342-6017-4177-8c56-c4091ebb98ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:25.217825 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72595944-dad6-42c2-92ae-5837595c6792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:25.219964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72595944-dad6-42c2-92ae-5837595c6792 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:25.220334 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72595944-dad6-42c2-92ae-5837595c6792 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:25.220769 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72595944-dad6-42c2-92ae-5837595c6792 admin admin] PATH_INFO: `/v3/projects/8966ccce76004a49b47282d73d52224f/users/ec6466270f474c149bb6c41dedccc2e5/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:25.244929 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72595944-dad6-42c2-92ae-5837595c6792 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8966ccce76004a49b47282d73d52224f, user_id=ec6466270f474c149bb6c41dedccc2e5, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:25.247801 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72595944-dad6-42c2-92ae-5837595c6792 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:25.259010 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51ba342-6017-4177-8c56-c4091ebb98ff admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:25.267158 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1004/2025] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:59:25 2022] PUT /identity/v3/projects/8966ccce76004a49b47282d73d52224f/users/ec6466270f474c149bb6c41dedccc2e5/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 14:59:25.284760 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf504c-6851-41ea-9312-560cbc9d532d None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:25.285114 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf504c-6851-41ea-9312-560cbc9d532d None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:25.285336 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf504c-6851-41ea-9312-560cbc9d532d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:25.317292 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-bfaf504c-6851-41ea-9312-560cbc9d532d None None] MFA Rules not processed for user `ec6466270f474c149bb6c41dedccc2e5`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:25.318647 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-bfaf504c-6851-41ea-9312-560cbc9d532d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:25.320872 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f668f86f-aeee-45ba-9686-3e1309e51845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:25.321734 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f668f86f-aeee-45ba-9686-3e1309e51845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:25.322018 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f668f86f-aeee-45ba-9686-3e1309e51845 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:25.322295 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f668f86f-aeee-45ba-9686-3e1309e51845 admin admin] PATH_INFO: `/v3/users/fb7fbea7e0e34f01acd7482ebe4a76b0` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:25.323500 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f668f86f-aeee-45ba-9686-3e1309e51845 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb7fbea7e0e34f01acd7482ebe4a76b0)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:25.325503 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f668f86f-aeee-45ba-9686-3e1309e51845 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:25.392764 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1005/2026] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:25.396303 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f668f86f-aeee-45ba-9686-3e1309e51845 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb7fbea7e0e34f01acd7482ebe4a76b0'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:25.404242 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaf504c-6851-41ea-9312-560cbc9d532d None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:25.405969 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-bfaf504c-6851-41ea-9312-560cbc9d532d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:25.406767 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f668f86f-aeee-45ba-9686-3e1309e51845 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb7fbea7e0e34f01acd7482ebe4a76b0'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:25.420264 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f668f86f-aeee-45ba-9686-3e1309e51845 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb7fbea7e0e34f01acd7482ebe4a76b0'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:25.421456 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f668f86f-aeee-45ba-9686-3e1309e51845 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb7fbea7e0e34f01acd7482ebe4a76b0'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:25.422443 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f668f86f-aeee-45ba-9686-3e1309e51845 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb7fbea7e0e34f01acd7482ebe4a76b0'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:25.450553 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1022/2027] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 14:59:25 2022] DELETE /identity/v3/users/fb7fbea7e0e34f01acd7482ebe4a76b0 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 14:59:25.472765 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaf504c-6851-41ea-9312-560cbc9d532d None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:25.537165 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f8eed6-3282-48a9-ab89-c7cda1a3a185 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:25.537824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8eed6-3282-48a9-ab89-c7cda1a3a185 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:25.538107 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8eed6-3282-48a9-ab89-c7cda1a3a185 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:25.538383 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8eed6-3282-48a9-ab89-c7cda1a3a185 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:25.540386 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-44f8eed6-3282-48a9-ab89-c7cda1a3a185 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:25.608526 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f8eed6-3282-48a9-ab89-c7cda1a3a185 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:25.610031 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f8eed6-3282-48a9-ab89-c7cda1a3a185 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:25.621839 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1006/2028] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:59:25 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 14:59:25.746809 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f668f86f-aeee-45ba-9686-3e1309e51845 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:25.748164 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f668f86f-aeee-45ba-9686-3e1309e51845 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:25.832319 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f668f86f-aeee-45ba-9686-3e1309e51845 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:25.843993 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84575bb1-1942-4833-9115-b802ae862ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:25.845576 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84575bb1-1942-4833-9115-b802ae862ab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:25.846138 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84575bb1-1942-4833-9115-b802ae862ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:25.846703 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84575bb1-1942-4833-9115-b802ae862ab8 admin admin] PATH_INFO: `/v3/projects/cc3f1bc686034eef92c670510f400b64` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:25.849633 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84575bb1-1942-4833-9115-b802ae862ab8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc3f1bc686034eef92c670510f400b64)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:25.852530 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84575bb1-1942-4833-9115-b802ae862ab8 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:25.905356 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-84575bb1-1942-4833-9115-b802ae862ab8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc3f1bc686034eef92c670510f400b64', 'request_id': 'req-84575bb1-1942-4833-9115-b802ae862ab8'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:25.911805 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-84575bb1-1942-4833-9115-b802ae862ab8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc3f1bc686034eef92c670510f400b64', 'request_id': 'req-84575bb1-1942-4833-9115-b802ae862ab8'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:25.938865 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-84575bb1-1942-4833-9115-b802ae862ab8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc3f1bc686034eef92c670510f400b64', 'request_id': 'req-84575bb1-1942-4833-9115-b802ae862ab8'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:25.952258 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-84575bb1-1942-4833-9115-b802ae862ab8 admin admin] The token cache is being invalidate because project cc3f1bc686034eef92c670510f400b64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 14:59:25.952554 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-84575bb1-1942-4833-9115-b802ae862ab8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:25.953323 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-84575bb1-1942-4833-9115-b802ae862ab8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:25.959954 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1023/2029] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 14:59:25 2022] DELETE /identity/v3/projects/cc3f1bc686034eef92c670510f400b64 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:25.962294 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e5db52-50c5-4e09-a17c-8fdcc4fff8fb None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:25.962593 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e5db52-50c5-4e09-a17c-8fdcc4fff8fb None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:25.964417 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e5db52-50c5-4e09-a17c-8fdcc4fff8fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:26.009685 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-88e5db52-50c5-4e09-a17c-8fdcc4fff8fb None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:26.010924 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-88e5db52-50c5-4e09-a17c-8fdcc4fff8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:26.117096 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1007/2030] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:26.126261 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f117a2-9d6a-4d09-9ff8-4342f37ffda7 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:26.126695 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f117a2-9d6a-4d09-9ff8-4342f37ffda7 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:26.126821 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f117a2-9d6a-4d09-9ff8-4342f37ffda7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:26.156838 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-c7f117a2-9d6a-4d09-9ff8-4342f37ffda7 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:26.158075 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-c7f117a2-9d6a-4d09-9ff8-4342f37ffda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:26.171823 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1024/2031] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:26.175916 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e5db52-50c5-4e09-a17c-8fdcc4fff8fb None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:26.188842 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e5db52-50c5-4e09-a17c-8fdcc4fff8fb None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:26.193906 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe7156-18e2-4e3f-a99d-f168b9ef72bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:26.194673 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe7156-18e2-4e3f-a99d-f168b9ef72bc admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:26.196108 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe7156-18e2-4e3f-a99d-f168b9ef72bc admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:26.196108 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe7156-18e2-4e3f-a99d-f168b9ef72bc admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:26.196526 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbe7156-18e2-4e3f-a99d-f168b9ef72bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:59:26.196862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbe7156-18e2-4e3f-a99d-f168b9ef72bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:26.198581 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbe7156-18e2-4e3f-a99d-f168b9ef72bc admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:26.213899 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1008/2032] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 14:59:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:59:26.218140 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f117a2-9d6a-4d09-9ff8-4342f37ffda7 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:26.228714 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f117a2-9d6a-4d09-9ff8-4342f37ffda7 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:26.235614 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea9a2e-a781-47c8-b889-dcf3a425f031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:26.236653 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea9a2e-a781-47c8-b889-dcf3a425f031 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:26.236653 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea9a2e-a781-47c8-b889-dcf3a425f031 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:26.237426 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea9a2e-a781-47c8-b889-dcf3a425f031 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:26.237631 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebea9a2e-a781-47c8-b889-dcf3a425f031 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:26.239732 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebea9a2e-a781-47c8-b889-dcf3a425f031 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:26.270119 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1025/2033] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:59:26 2022] POST /identity/v3/projects => generated 397 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 14:59:26.276098 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe7156-18e2-4e3f-a99d-f168b9ef72bc admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:26.284960 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe7156-18e2-4e3f-a99d-f168b9ef72bc admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:26.374597 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa91757-f42a-4d26-b1d6-766d909857f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:26.375535 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa91757-f42a-4d26-b1d6-766d909857f6 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:26.375824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa91757-f42a-4d26-b1d6-766d909857f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:26.376100 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa91757-f42a-4d26-b1d6-766d909857f6 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:26.377055 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa91757-f42a-4d26-b1d6-766d909857f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:26.379730 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa91757-f42a-4d26-b1d6-766d909857f6 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:26.400722 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1009/2034] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:59:26 2022] POST /identity/v3/users => generated 299 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:59:26.405025 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea9a2e-a781-47c8-b889-dcf3a425f031 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:26.416225 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea9a2e-a781-47c8-b889-dcf3a425f031 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:26.422766 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e300e1-2c14-4929-997a-6a3e0bc4a563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:26.423444 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e300e1-2c14-4929-997a-6a3e0bc4a563 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:26.423729 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e300e1-2c14-4929-997a-6a3e0bc4a563 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:26.424008 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e300e1-2c14-4929-997a-6a3e0bc4a563 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:26.424771 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e300e1-2c14-4929-997a-6a3e0bc4a563 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:26.426362 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e300e1-2c14-4929-997a-6a3e0bc4a563 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:26.440878 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1026/2035] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 06 14:59:26.442047 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa91757-f42a-4d26-b1d6-766d909857f6 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:26.452129 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa91757-f42a-4d26-b1d6-766d909857f6 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:26.456959 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05032bb1-353f-48ea-8eec-9f8180b1cc5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:26.457768 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05032bb1-353f-48ea-8eec-9f8180b1cc5c admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:26.458078 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05032bb1-353f-48ea-8eec-9f8180b1cc5c admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:26.458377 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05032bb1-353f-48ea-8eec-9f8180b1cc5c admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:26.459100 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05032bb1-353f-48ea-8eec-9f8180b1cc5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:26.461249 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05032bb1-353f-48ea-8eec-9f8180b1cc5c admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:26.471676 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1010/2036] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:59:26.480915 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e300e1-2c14-4929-997a-6a3e0bc4a563 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:26.489606 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e300e1-2c14-4929-997a-6a3e0bc4a563 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:26.494459 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda5946-ddd8-495b-a7b3-27c8830a1530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:26.495368 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda5946-ddd8-495b-a7b3-27c8830a1530 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:26.495636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda5946-ddd8-495b-a7b3-27c8830a1530 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:26.495911 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda5946-ddd8-495b-a7b3-27c8830a1530 admin admin] PATH_INFO: `/v3/projects/3cbab0e6e3e24a03b5b710c590759164/users/41599f480e344b509d4748881bd94991/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:26.513305 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cda5946-ddd8-495b-a7b3-27c8830a1530 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cbab0e6e3e24a03b5b710c590759164, user_id=41599f480e344b509d4748881bd94991, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:26.515970 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cda5946-ddd8-495b-a7b3-27c8830a1530 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:26.533317 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1027/2037] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:59:26 2022] PUT /identity/v3/projects/3cbab0e6e3e24a03b5b710c590759164/users/41599f480e344b509d4748881bd94991/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:26.545068 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684e5b6-7b6e-469b-b594-4c8851de31cf None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:26.545354 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684e5b6-7b6e-469b-b594-4c8851de31cf None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:26.545633 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684e5b6-7b6e-469b-b594-4c8851de31cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:26.582783 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-2684e5b6-7b6e-469b-b594-4c8851de31cf None None] MFA Rules not processed for user `41599f480e344b509d4748881bd94991`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:26.583995 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-2684e5b6-7b6e-469b-b594-4c8851de31cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:26.650207 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1011/2038] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:26.662685 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda5946-ddd8-495b-a7b3-27c8830a1530 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:26.665316 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-5cda5946-ddd8-495b-a7b3-27c8830a1530 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:26.720997 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda5946-ddd8-495b-a7b3-27c8830a1530 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:26.724570 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2be101d-900b-422d-ac77-45c2aabf29dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:26.725244 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be101d-900b-422d-ac77-45c2aabf29dc service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:26.725528 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be101d-900b-422d-ac77-45c2aabf29dc service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:26.725804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be101d-900b-422d-ac77-45c2aabf29dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:26.734961 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2be101d-900b-422d-ac77-45c2aabf29dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:26.736393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2be101d-900b-422d-ac77-45c2aabf29dc service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:26.745384 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1028/2039] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:59:26 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 14:59:43.285814 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2684e5b6-7b6e-469b-b594-4c8851de31cf None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:43.287564 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-2684e5b6-7b6e-469b-b594-4c8851de31cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:43.365973 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2684e5b6-7b6e-469b-b594-4c8851de31cf None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:43.372204 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46c71cb-3c28-42a1-83c5-34d172fa7c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:43.372204 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c71cb-3c28-42a1-83c5-34d172fa7c49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:43.372510 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c71cb-3c28-42a1-83c5-34d172fa7c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:43.372808 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c71cb-3c28-42a1-83c5-34d172fa7c49 admin admin] PATH_INFO: `/v3/users/19347c6703b74caebef5c713305945fd` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:43.374002 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46c71cb-3c28-42a1-83c5-34d172fa7c49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19347c6703b74caebef5c713305945fd)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:43.375930 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46c71cb-3c28-42a1-83c5-34d172fa7c49 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:43.437834 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-a46c71cb-3c28-42a1-83c5-34d172fa7c49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19347c6703b74caebef5c713305945fd'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:43.438496 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-a46c71cb-3c28-42a1-83c5-34d172fa7c49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19347c6703b74caebef5c713305945fd'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:43.448918 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-a46c71cb-3c28-42a1-83c5-34d172fa7c49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19347c6703b74caebef5c713305945fd'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:43.455773 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-a46c71cb-3c28-42a1-83c5-34d172fa7c49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19347c6703b74caebef5c713305945fd'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:43.456463 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-a46c71cb-3c28-42a1-83c5-34d172fa7c49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19347c6703b74caebef5c713305945fd'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:43.475470 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1012/2040] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 14:59:43 2022] DELETE /identity/v3/users/19347c6703b74caebef5c713305945fd => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:43.480169 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4480852-7985-42a2-b77c-026082e08fd4 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:43.480713 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4480852-7985-42a2-b77c-026082e08fd4 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:43.480828 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4480852-7985-42a2-b77c-026082e08fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:43.500696 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-c4480852-7985-42a2-b77c-026082e08fd4 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:43.501920 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-c4480852-7985-42a2-b77c-026082e08fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:43.570211 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1029/2041] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:43.581116 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46c71cb-3c28-42a1-83c5-34d172fa7c49 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:43.583262 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-a46c71cb-3c28-42a1-83c5-34d172fa7c49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:43.648790 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46c71cb-3c28-42a1-83c5-34d172fa7c49 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:43.654365 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058b38e-07ba-4566-bdeb-b436883b354e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:43.655202 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058b38e-07ba-4566-bdeb-b436883b354e service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:43.655840 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058b38e-07ba-4566-bdeb-b436883b354e service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:43.656203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058b38e-07ba-4566-bdeb-b436883b354e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:43.667858 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6058b38e-07ba-4566-bdeb-b436883b354e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:43.669684 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6058b38e-07ba-4566-bdeb-b436883b354e service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:43.680626 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1013/2042] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:59:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:59:43.978234 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4480852-7985-42a2-b77c-026082e08fd4 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:43.980969 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-c4480852-7985-42a2-b77c-026082e08fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:43.995932 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4480852-7985-42a2-b77c-026082e08fd4 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:44.001623 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea4f088-30da-41f9-b2df-dd22811776c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:44.003288 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4f088-30da-41f9-b2df-dd22811776c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:44.003587 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4f088-30da-41f9-b2df-dd22811776c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:44.003932 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4f088-30da-41f9-b2df-dd22811776c8 admin admin] PATH_INFO: `/v3/projects/170aa3ef89474632955b143b4665b6e6` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:44.005465 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea4f088-30da-41f9-b2df-dd22811776c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=170aa3ef89474632955b143b4665b6e6)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:44.007736 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea4f088-30da-41f9-b2df-dd22811776c8 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:44.068894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-dea4f088-30da-41f9-b2df-dd22811776c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '170aa3ef89474632955b143b4665b6e6', 'request_id': 'req-dea4f088-30da-41f9-b2df-dd22811776c8'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:44.075814 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-dea4f088-30da-41f9-b2df-dd22811776c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '170aa3ef89474632955b143b4665b6e6', 'request_id': 'req-dea4f088-30da-41f9-b2df-dd22811776c8'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:44.085194 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-dea4f088-30da-41f9-b2df-dd22811776c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '170aa3ef89474632955b143b4665b6e6', 'request_id': 'req-dea4f088-30da-41f9-b2df-dd22811776c8'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:44.097650 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-dea4f088-30da-41f9-b2df-dd22811776c8 admin admin] The token cache is being invalidate because project 170aa3ef89474632955b143b4665b6e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 14:59:44.098083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-dea4f088-30da-41f9-b2df-dd22811776c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:44.098977 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-dea4f088-30da-41f9-b2df-dd22811776c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:44.100636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1030/2043] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 14:59:43 2022] DELETE /identity/v3/projects/170aa3ef89474632955b143b4665b6e6 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:44.106754 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a57b6e-4706-4840-8881-ec905bbecf93 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:44.107039 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a57b6e-4706-4840-8881-ec905bbecf93 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:44.107434 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a57b6e-4706-4840-8881-ec905bbecf93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:44.127269 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-75a57b6e-4706-4840-8881-ec905bbecf93 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:44.128468 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-75a57b6e-4706-4840-8881-ec905bbecf93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:44.199598 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea4f088-30da-41f9-b2df-dd22811776c8 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:44.207016 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1014/2044] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 14:59:44.207112 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-dea4f088-30da-41f9-b2df-dd22811776c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:44.208657 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17fd94-a7a3-4a1c-9976-526caafb684b None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:44.208956 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17fd94-a7a3-4a1c-9976-526caafb684b None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:44.209238 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17fd94-a7a3-4a1c-9976-526caafb684b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:44.222293 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea4f088-30da-41f9-b2df-dd22811776c8 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:44.230808 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb99539-c818-4bfd-b095-4b792b6976ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:44.232207 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb99539-c818-4bfd-b095-4b792b6976ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:44.232941 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb99539-c818-4bfd-b095-4b792b6976ee admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:44.233492 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb99539-c818-4bfd-b095-4b792b6976ee admin admin] PATH_INFO: `/v3/users/41599f480e344b509d4748881bd94991` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:44.248226 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb99539-c818-4bfd-b095-4b792b6976ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=41599f480e344b509d4748881bd94991)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:44.248413 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-ae17fd94-a7a3-4a1c-9976-526caafb684b None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:44.250584 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ae17fd94-a7a3-4a1c-9976-526caafb684b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:44.250723 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb99539-c818-4bfd-b095-4b792b6976ee admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:44.264706 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1015/2045] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:44.265782 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae17fd94-a7a3-4a1c-9976-526caafb684b None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:44.277083 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae17fd94-a7a3-4a1c-9976-526caafb684b None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:44.284087 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ca591-6897-4d9f-863d-d9f5651905bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:44.284801 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ca591-6897-4d9f-863d-d9f5651905bc admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:44.285090 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ca591-6897-4d9f-863d-d9f5651905bc admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:44.285364 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ca591-6897-4d9f-863d-d9f5651905bc admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:44.286112 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ca591-6897-4d9f-863d-d9f5651905bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:59:44.286713 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ca591-6897-4d9f-863d-d9f5651905bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:44.288033 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ca591-6897-4d9f-863d-d9f5651905bc admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:44.301529 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1016/2046] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 14:59:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:59:44.306350 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ca591-6897-4d9f-863d-d9f5651905bc admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:44.318033 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ca591-6897-4d9f-863d-d9f5651905bc admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:44.323469 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3932577-e3f2-469c-b9f2-3ad470dedf6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:44.324286 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3932577-e3f2-469c-b9f2-3ad470dedf6a admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:44.324593 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3932577-e3f2-469c-b9f2-3ad470dedf6a admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:44.324907 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3932577-e3f2-469c-b9f2-3ad470dedf6a admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:44.325580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3932577-e3f2-469c-b9f2-3ad470dedf6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:44.327517 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3932577-e3f2-469c-b9f2-3ad470dedf6a admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:44.332819 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-6eb99539-c818-4bfd-b095-4b792b6976ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41599f480e344b509d4748881bd94991'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:44.342200 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-6eb99539-c818-4bfd-b095-4b792b6976ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41599f480e344b509d4748881bd94991'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:44.353075 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-6eb99539-c818-4bfd-b095-4b792b6976ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41599f480e344b509d4748881bd94991'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:44.353841 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-6eb99539-c818-4bfd-b095-4b792b6976ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41599f480e344b509d4748881bd94991'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:44.355175 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-6eb99539-c818-4bfd-b095-4b792b6976ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41599f480e344b509d4748881bd94991'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:44.358636 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1017/2047] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:59:44 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:59:44.366035 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3932577-e3f2-469c-b9f2-3ad470dedf6a admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:44.367825 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-f3932577-e3f2-469c-b9f2-3ad470dedf6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:44.383350 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1031/2048] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 14:59:44 2022] DELETE /identity/v3/users/41599f480e344b509d4748881bd94991 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 14:59:44.385643 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c0129-d7e1-4133-b7d1-efdb01754246 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:44.385920 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c0129-d7e1-4133-b7d1-efdb01754246 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:44.386285 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c0129-d7e1-4133-b7d1-efdb01754246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:44.409356 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-556c0129-d7e1-4133-b7d1-efdb01754246 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:44.410660 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-556c0129-d7e1-4133-b7d1-efdb01754246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:44.455498 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3932577-e3f2-469c-b9f2-3ad470dedf6a admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:44.477055 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1032/2049] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:44.487498 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556c0129-d7e1-4133-b7d1-efdb01754246 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:44.489787 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-556c0129-d7e1-4133-b7d1-efdb01754246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:44.521213 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055474cb-27cb-4da2-9fb2-80a66489f473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:44.522291 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055474cb-27cb-4da2-9fb2-80a66489f473 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:44.522655 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055474cb-27cb-4da2-9fb2-80a66489f473 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:44.523005 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055474cb-27cb-4da2-9fb2-80a66489f473 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:44.523781 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055474cb-27cb-4da2-9fb2-80a66489f473 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:44.525998 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055474cb-27cb-4da2-9fb2-80a66489f473 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:44.557247 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1018/2050] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:59:44 2022] POST /identity/v3/users => generated 292 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:59:44.561372 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055474cb-27cb-4da2-9fb2-80a66489f473 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:44.566313 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556c0129-d7e1-4133-b7d1-efdb01754246 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:44.570063 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eba20b-0537-41f9-9ad7-61e99824cbf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:44.570880 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eba20b-0537-41f9-9ad7-61e99824cbf5 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:44.571258 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eba20b-0537-41f9-9ad7-61e99824cbf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:44.571633 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eba20b-0537-41f9-9ad7-61e99824cbf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:44.576616 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055474cb-27cb-4da2-9fb2-80a66489f473 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:44.584323 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eba20b-0537-41f9-9ad7-61e99824cbf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:44.585979 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eba20b-0537-41f9-9ad7-61e99824cbf5 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:44.586821 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca94b47c-9a74-47e9-9167-cbde3af0982a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:44.587506 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca94b47c-9a74-47e9-9167-cbde3af0982a admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:44.587797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca94b47c-9a74-47e9-9167-cbde3af0982a admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:44.588073 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca94b47c-9a74-47e9-9167-cbde3af0982a admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:44.589490 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca94b47c-9a74-47e9-9167-cbde3af0982a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:44.591288 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca94b47c-9a74-47e9-9167-cbde3af0982a admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:44.599043 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1033/2051] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:59:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 14:59:44.605355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1019/2052] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:44 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 06 14:59:44.609322 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eba20b-0537-41f9-9ad7-61e99824cbf5 service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:44.619963 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eba20b-0537-41f9-9ad7-61e99824cbf5 service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:44.626736 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abd7389-38b7-4c04-999b-8a009fc65f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:44.627417 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd7389-38b7-4c04-999b-8a009fc65f0e admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:44.627709 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd7389-38b7-4c04-999b-8a009fc65f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:44.627977 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd7389-38b7-4c04-999b-8a009fc65f0e admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:44.628732 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abd7389-38b7-4c04-999b-8a009fc65f0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:44.630336 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abd7389-38b7-4c04-999b-8a009fc65f0e admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:44.644320 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1034/2053] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:59:44.645816 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca94b47c-9a74-47e9-9167-cbde3af0982a admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:44.656866 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca94b47c-9a74-47e9-9167-cbde3af0982a admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:44.661916 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa1db7-d631-4d04-81f7-8bfacca5836c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:44.664166 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa1db7-d631-4d04-81f7-8bfacca5836c admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:44.664166 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa1db7-d631-4d04-81f7-8bfacca5836c admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:44.664166 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa1db7-d631-4d04-81f7-8bfacca5836c admin admin] PATH_INFO: `/v3/projects/fda6209766de499ab769f0a7da4bb412/users/28ef9b8939d84f83a4bc44e393f27bd5/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:44.681931 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaa1db7-d631-4d04-81f7-8bfacca5836c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fda6209766de499ab769f0a7da4bb412, user_id=28ef9b8939d84f83a4bc44e393f27bd5, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:44.685245 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaa1db7-d631-4d04-81f7-8bfacca5836c admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:44.701270 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1020/2054] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:59:44 2022] PUT /identity/v3/projects/fda6209766de499ab769f0a7da4bb412/users/28ef9b8939d84f83a4bc44e393f27bd5/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:44.714255 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77caeb-9bd4-43ae-9f23-17ebae8f6e5f None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:44.714453 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77caeb-9bd4-43ae-9f23-17ebae8f6e5f None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:44.714721 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77caeb-9bd4-43ae-9f23-17ebae8f6e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:44.748292 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-2e77caeb-9bd4-43ae-9f23-17ebae8f6e5f None None] MFA Rules not processed for user `28ef9b8939d84f83a4bc44e393f27bd5`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:44.749566 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-2e77caeb-9bd4-43ae-9f23-17ebae8f6e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:44.835825 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1035/2055] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:44.849282 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa1db7-d631-4d04-81f7-8bfacca5836c admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:44.851156 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-caaa1db7-d631-4d04-81f7-8bfacca5836c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:44.932890 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa1db7-d631-4d04-81f7-8bfacca5836c admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:44.938548 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212d2e4b-336a-44a1-bf41-c89f01ed4864 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:44.939282 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d2e4b-336a-44a1-bf41-c89f01ed4864 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:44.939581 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d2e4b-336a-44a1-bf41-c89f01ed4864 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:44.939953 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d2e4b-336a-44a1-bf41-c89f01ed4864 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:44.955429 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212d2e4b-336a-44a1-bf41-c89f01ed4864 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:44.957149 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212d2e4b-336a-44a1-bf41-c89f01ed4864 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:44.965781 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e77caeb-9bd4-43ae-9f23-17ebae8f6e5f None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:44.967733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-2e77caeb-9bd4-43ae-9f23-17ebae8f6e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:44.985059 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1021/2056] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:59:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:59:45.061366 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e77caeb-9bd4-43ae-9f23-17ebae8f6e5f None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:45.067262 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33296c5c-7cda-4701-ab4c-1f28cc519a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:45.068135 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33296c5c-7cda-4701-ab4c-1f28cc519a27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:45.068410 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33296c5c-7cda-4701-ab4c-1f28cc519a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:45.068711 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33296c5c-7cda-4701-ab4c-1f28cc519a27 admin admin] PATH_INFO: `/v3/projects/3cbab0e6e3e24a03b5b710c590759164` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:45.070467 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33296c5c-7cda-4701-ab4c-1f28cc519a27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cbab0e6e3e24a03b5b710c590759164)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:45.072460 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33296c5c-7cda-4701-ab4c-1f28cc519a27 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:45.137468 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-33296c5c-7cda-4701-ab4c-1f28cc519a27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cbab0e6e3e24a03b5b710c590759164', 'request_id': 'req-33296c5c-7cda-4701-ab4c-1f28cc519a27'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:45.144324 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-33296c5c-7cda-4701-ab4c-1f28cc519a27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cbab0e6e3e24a03b5b710c590759164', 'request_id': 'req-33296c5c-7cda-4701-ab4c-1f28cc519a27'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:45.165811 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-33296c5c-7cda-4701-ab4c-1f28cc519a27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cbab0e6e3e24a03b5b710c590759164', 'request_id': 'req-33296c5c-7cda-4701-ab4c-1f28cc519a27'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:45.180166 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-33296c5c-7cda-4701-ab4c-1f28cc519a27 admin admin] The token cache is being invalidate because project 3cbab0e6e3e24a03b5b710c590759164 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 14:59:45.180457 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-33296c5c-7cda-4701-ab4c-1f28cc519a27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:45.181133 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-33296c5c-7cda-4701-ab4c-1f28cc519a27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:45.183598 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1036/2057] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 14:59:44 2022] DELETE /identity/v3/projects/3cbab0e6e3e24a03b5b710c590759164 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:45.189572 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0b287-8e50-40af-874c-d66a09cc8532 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:45.189898 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0b287-8e50-40af-874c-d66a09cc8532 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:45.190130 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0b287-8e50-40af-874c-d66a09cc8532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:45.215211 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-b9d0b287-8e50-40af-874c-d66a09cc8532 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:45.216560 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b9d0b287-8e50-40af-874c-d66a09cc8532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:45.302394 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1022/2058] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:45.312125 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b674fad-3f20-4bc2-a4db-6ad062b2c763 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:45.312492 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b674fad-3f20-4bc2-a4db-6ad062b2c763 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:45.313263 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b674fad-3f20-4bc2-a4db-6ad062b2c763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:45.340654 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-8b674fad-3f20-4bc2-a4db-6ad062b2c763 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:45.343532 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-8b674fad-3f20-4bc2-a4db-6ad062b2c763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:45.376451 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1037/2059] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 14:59:45.380441 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d0b287-8e50-40af-874c-d66a09cc8532 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:45.396975 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d0b287-8e50-40af-874c-d66a09cc8532 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:45.409960 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f0356f-6b3c-45d0-be2b-e2b453047f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:45.410882 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0356f-6b3c-45d0-be2b-e2b453047f5c admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:45.411918 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0356f-6b3c-45d0-be2b-e2b453047f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:45.412386 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0356f-6b3c-45d0-be2b-e2b453047f5c admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:45.413316 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f0356f-6b3c-45d0-be2b-e2b453047f5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:59:45.413630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f0356f-6b3c-45d0-be2b-e2b453047f5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:45.415250 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f0356f-6b3c-45d0-be2b-e2b453047f5c admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:45.442486 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1023/2060] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 14:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 14:59:45.447172 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b674fad-3f20-4bc2-a4db-6ad062b2c763 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:45.475818 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b674fad-3f20-4bc2-a4db-6ad062b2c763 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:45.482595 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a897c7f8-acc1-4dc2-a086-240bfd99387d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:45.483422 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a897c7f8-acc1-4dc2-a086-240bfd99387d admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:45.483863 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a897c7f8-acc1-4dc2-a086-240bfd99387d admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:45.484138 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a897c7f8-acc1-4dc2-a086-240bfd99387d admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:45.484923 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a897c7f8-acc1-4dc2-a086-240bfd99387d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:45.487310 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a897c7f8-acc1-4dc2-a086-240bfd99387d admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:45.520780 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1038/2061] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:59:45 2022] POST /identity/v3/projects => generated 379 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:59:45.525286 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f0356f-6b3c-45d0-be2b-e2b453047f5c admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:45.537270 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f0356f-6b3c-45d0-be2b-e2b453047f5c admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:45.620225 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32defe64-0295-4d67-9ef8-a41957ee8b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:45.621166 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32defe64-0295-4d67-9ef8-a41957ee8b8d admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:45.621453 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32defe64-0295-4d67-9ef8-a41957ee8b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:45.621772 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32defe64-0295-4d67-9ef8-a41957ee8b8d admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:45.622532 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32defe64-0295-4d67-9ef8-a41957ee8b8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:45.624909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32defe64-0295-4d67-9ef8-a41957ee8b8d admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:45.659885 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1024/2062] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:59:45 2022] POST /identity/v3/users => generated 290 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:59:45.665871 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a897c7f8-acc1-4dc2-a086-240bfd99387d admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:45.676866 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a897c7f8-acc1-4dc2-a086-240bfd99387d admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:45.683599 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac6842b-6299-4e68-bc3c-ec270525c805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:45.684281 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac6842b-6299-4e68-bc3c-ec270525c805 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:45.684567 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac6842b-6299-4e68-bc3c-ec270525c805 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:45.685055 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac6842b-6299-4e68-bc3c-ec270525c805 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:45.685800 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac6842b-6299-4e68-bc3c-ec270525c805 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:45.687714 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac6842b-6299-4e68-bc3c-ec270525c805 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:45.701273 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1039/2063] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:59:45.703951 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32defe64-0295-4d67-9ef8-a41957ee8b8d admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:45.715098 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32defe64-0295-4d67-9ef8-a41957ee8b8d admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:45.722007 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b935111-bb4c-4a4b-b19e-d940752e12ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:45.722817 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b935111-bb4c-4a4b-b19e-d940752e12ef admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:45.723202 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b935111-bb4c-4a4b-b19e-d940752e12ef admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:45.723790 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b935111-bb4c-4a4b-b19e-d940752e12ef admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:45.724757 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b935111-bb4c-4a4b-b19e-d940752e12ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:45.731232 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b935111-bb4c-4a4b-b19e-d940752e12ef admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:45.744843 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1025/2064] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:59:45.746894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac6842b-6299-4e68-bc3c-ec270525c805 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:45.759143 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac6842b-6299-4e68-bc3c-ec270525c805 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:45.767193 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f5bdfe-ec6f-4e55-a516-86104267e727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:45.769445 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5bdfe-ec6f-4e55-a516-86104267e727 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:45.769903 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5bdfe-ec6f-4e55-a516-86104267e727 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:45.770453 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5bdfe-ec6f-4e55-a516-86104267e727 admin admin] PATH_INFO: `/v3/projects/bb5f41fdaad246b7bc30c8f8021129a3/users/4b3af1d82c33446e998f07bce54ef941/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:45.806058 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f5bdfe-ec6f-4e55-a516-86104267e727 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb5f41fdaad246b7bc30c8f8021129a3, user_id=4b3af1d82c33446e998f07bce54ef941, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:45.810341 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f5bdfe-ec6f-4e55-a516-86104267e727 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:45.827465 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1040/2065] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:59:45 2022] PUT /identity/v3/projects/bb5f41fdaad246b7bc30c8f8021129a3/users/4b3af1d82c33446e998f07bce54ef941/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:45.832314 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b597e-63e0-48da-b756-f8eb49397183 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:45.832502 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b597e-63e0-48da-b756-f8eb49397183 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:45.832825 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b597e-63e0-48da-b756-f8eb49397183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:45.852568 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-3c5b597e-63e0-48da-b756-f8eb49397183 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:45.853861 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-3c5b597e-63e0-48da-b756-f8eb49397183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:45.919758 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1026/2066] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:45.929538 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f5bdfe-ec6f-4e55-a516-86104267e727 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:45.931490 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-83f5bdfe-ec6f-4e55-a516-86104267e727 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:45.988838 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f5bdfe-ec6f-4e55-a516-86104267e727 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:45.992650 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7087077a-2c5d-4ea1-a91b-cbd3b59ca3e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:45.993284 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087077a-2c5d-4ea1-a91b-cbd3b59ca3e9 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:45.993567 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087077a-2c5d-4ea1-a91b-cbd3b59ca3e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:45.993837 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087077a-2c5d-4ea1-a91b-cbd3b59ca3e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:46.003535 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7087077a-2c5d-4ea1-a91b-cbd3b59ca3e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:46.005027 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7087077a-2c5d-4ea1-a91b-cbd3b59ca3e9 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:46.015759 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1041/2067] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:59:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:59:47.648469 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b597e-63e0-48da-b756-f8eb49397183 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:47.650623 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-3c5b597e-63e0-48da-b756-f8eb49397183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:47.670671 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b597e-63e0-48da-b756-f8eb49397183 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:47.676178 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e886a-c1d3-485f-b6a9-38e6b37299ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:47.677140 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e886a-c1d3-485f-b6a9-38e6b37299ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:47.677565 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e886a-c1d3-485f-b6a9-38e6b37299ef admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:47.677972 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e886a-c1d3-485f-b6a9-38e6b37299ef admin admin] PATH_INFO: `/v3/users/28ef9b8939d84f83a4bc44e393f27bd5` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:47.679834 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1e886a-c1d3-485f-b6a9-38e6b37299ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=28ef9b8939d84f83a4bc44e393f27bd5)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:47.684841 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1e886a-c1d3-485f-b6a9-38e6b37299ef admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:47.764325 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-dc1e886a-c1d3-485f-b6a9-38e6b37299ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28ef9b8939d84f83a4bc44e393f27bd5'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:47.765060 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-dc1e886a-c1d3-485f-b6a9-38e6b37299ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28ef9b8939d84f83a4bc44e393f27bd5'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:47.781957 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-dc1e886a-c1d3-485f-b6a9-38e6b37299ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28ef9b8939d84f83a4bc44e393f27bd5'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:47.792821 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-dc1e886a-c1d3-485f-b6a9-38e6b37299ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28ef9b8939d84f83a4bc44e393f27bd5'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:47.794320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-dc1e886a-c1d3-485f-b6a9-38e6b37299ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28ef9b8939d84f83a4bc44e393f27bd5'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:47.825625 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1027/2068] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 14:59:47 2022] DELETE /identity/v3/users/28ef9b8939d84f83a4bc44e393f27bd5 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:47.830115 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0743cf-d3b8-454a-b1be-337ed4da52ad None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:47.830572 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0743cf-d3b8-454a-b1be-337ed4da52ad None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:47.830680 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0743cf-d3b8-454a-b1be-337ed4da52ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:47.865543 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-4f0743cf-d3b8-454a-b1be-337ed4da52ad None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:47.867023 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-4f0743cf-d3b8-454a-b1be-337ed4da52ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:47.954892 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1042/2069] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:47.979340 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e886a-c1d3-485f-b6a9-38e6b37299ef admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:47.983403 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-dc1e886a-c1d3-485f-b6a9-38e6b37299ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:48.117268 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e886a-c1d3-485f-b6a9-38e6b37299ef admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:48.121329 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d12b3-ecc9-4dcd-8010-43f72bb836ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:48.121972 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d12b3-ecc9-4dcd-8010-43f72bb836ed service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:48.122254 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d12b3-ecc9-4dcd-8010-43f72bb836ed service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:48.122589 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d12b3-ecc9-4dcd-8010-43f72bb836ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:48.135627 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3d12b3-ecc9-4dcd-8010-43f72bb836ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:48.138422 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3d12b3-ecc9-4dcd-8010-43f72bb836ed service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:48.172979 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1028/2070] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:59:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:59:49.276921 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0743cf-d3b8-454a-b1be-337ed4da52ad None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:49.278598 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-4f0743cf-d3b8-454a-b1be-337ed4da52ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:49.290861 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0743cf-d3b8-454a-b1be-337ed4da52ad None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:49.296680 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eabb9b0-0386-47de-afca-535b746fed51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:49.297806 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eabb9b0-0386-47de-afca-535b746fed51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:49.298267 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eabb9b0-0386-47de-afca-535b746fed51 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:49.298716 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eabb9b0-0386-47de-afca-535b746fed51 admin admin] PATH_INFO: `/v3/projects/fda6209766de499ab769f0a7da4bb412` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:49.299945 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eabb9b0-0386-47de-afca-535b746fed51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fda6209766de499ab769f0a7da4bb412)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:49.301908 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eabb9b0-0386-47de-afca-535b746fed51 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:49.385094 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-8eabb9b0-0386-47de-afca-535b746fed51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fda6209766de499ab769f0a7da4bb412', 'request_id': 'req-8eabb9b0-0386-47de-afca-535b746fed51'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:49.402354 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-8eabb9b0-0386-47de-afca-535b746fed51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fda6209766de499ab769f0a7da4bb412', 'request_id': 'req-8eabb9b0-0386-47de-afca-535b746fed51'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:49.412213 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-8eabb9b0-0386-47de-afca-535b746fed51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fda6209766de499ab769f0a7da4bb412', 'request_id': 'req-8eabb9b0-0386-47de-afca-535b746fed51'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:49.426784 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-8eabb9b0-0386-47de-afca-535b746fed51 admin admin] The token cache is being invalidate because project fda6209766de499ab769f0a7da4bb412 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 14:59:49.427380 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-8eabb9b0-0386-47de-afca-535b746fed51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:49.428216 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-8eabb9b0-0386-47de-afca-535b746fed51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:49.430657 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1043/2071] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 14:59:49 2022] DELETE /identity/v3/projects/fda6209766de499ab769f0a7da4bb412 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:49.436063 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd0a8f-4ebb-4628-b418-56949e93c282 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:49.436714 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd0a8f-4ebb-4628-b418-56949e93c282 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:49.436714 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd0a8f-4ebb-4628-b418-56949e93c282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:49.485803 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-0abd0a8f-4ebb-4628-b418-56949e93c282 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:49.487035 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-0abd0a8f-4ebb-4628-b418-56949e93c282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:49.649869 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1029/2072] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:49.664539 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb545c-d079-43fd-8d78-212a40c15ea4 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:49.664539 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb545c-d079-43fd-8d78-212a40c15ea4 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:49.664539 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb545c-d079-43fd-8d78-212a40c15ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:49.688746 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-1aeb545c-d079-43fd-8d78-212a40c15ea4 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:49.689976 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-1aeb545c-d079-43fd-8d78-212a40c15ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:49.700964 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1044/2073] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:49.705338 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd0a8f-4ebb-4628-b418-56949e93c282 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:49.716493 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd0a8f-4ebb-4628-b418-56949e93c282 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:49.722057 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4f9a1f-0ca3-40df-ab99-3892ac9df434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:49.722727 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f9a1f-0ca3-40df-ab99-3892ac9df434 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:49.723008 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f9a1f-0ca3-40df-ab99-3892ac9df434 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:49.723284 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f9a1f-0ca3-40df-ab99-3892ac9df434 admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:49.724015 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4f9a1f-0ca3-40df-ab99-3892ac9df434 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:59:49.724313 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4f9a1f-0ca3-40df-ab99-3892ac9df434 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:49.726170 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4f9a1f-0ca3-40df-ab99-3892ac9df434 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:49.739650 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1030/2074] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 14:59:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:59:49.744175 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeb545c-d079-43fd-8d78-212a40c15ea4 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:49.762048 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeb545c-d079-43fd-8d78-212a40c15ea4 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:49.801955 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6275d02-c4a9-4fb8-8bd8-010b69b2611e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:49.804343 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6275d02-c4a9-4fb8-8bd8-010b69b2611e admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:49.804932 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6275d02-c4a9-4fb8-8bd8-010b69b2611e admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:49.805561 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6275d02-c4a9-4fb8-8bd8-010b69b2611e admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:49.808961 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6275d02-c4a9-4fb8-8bd8-010b69b2611e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:49.817963 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6275d02-c4a9-4fb8-8bd8-010b69b2611e admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:49.863386 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1045/2075] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:59:49 2022] POST /identity/v3/projects => generated 365 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 14:59:49.865226 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4f9a1f-0ca3-40df-ab99-3892ac9df434 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:49.881815 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4f9a1f-0ca3-40df-ab99-3892ac9df434 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:49.986438 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a61e5b-0e3f-4948-9f08-a1ebb1d007b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:49.987376 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a61e5b-0e3f-4948-9f08-a1ebb1d007b2 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:49.987660 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a61e5b-0e3f-4948-9f08-a1ebb1d007b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:49.988221 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a61e5b-0e3f-4948-9f08-a1ebb1d007b2 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:49.988816 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a61e5b-0e3f-4948-9f08-a1ebb1d007b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:49.990805 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a61e5b-0e3f-4948-9f08-a1ebb1d007b2 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:50.016544 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1031/2076] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:59:49 2022] POST /identity/v3/users => generated 283 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:59:50.021453 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6275d02-c4a9-4fb8-8bd8-010b69b2611e admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:50.036620 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6275d02-c4a9-4fb8-8bd8-010b69b2611e admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:50.046031 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29893181-2d02-4fb9-b0b1-fd3b14142cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:50.047185 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29893181-2d02-4fb9-b0b1-fd3b14142cd8 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:50.047488 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29893181-2d02-4fb9-b0b1-fd3b14142cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:50.047750 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29893181-2d02-4fb9-b0b1-fd3b14142cd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:50.049539 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29893181-2d02-4fb9-b0b1-fd3b14142cd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:50.051342 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29893181-2d02-4fb9-b0b1-fd3b14142cd8 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:50.069566 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1046/2077] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:59:50.073341 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a61e5b-0e3f-4948-9f08-a1ebb1d007b2 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:50.088068 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a61e5b-0e3f-4948-9f08-a1ebb1d007b2 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:50.096195 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914bf3d9-422c-426a-ba34-2a95799f4353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:50.097144 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914bf3d9-422c-426a-ba34-2a95799f4353 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:50.097494 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914bf3d9-422c-426a-ba34-2a95799f4353 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:50.098575 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914bf3d9-422c-426a-ba34-2a95799f4353 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:50.099421 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914bf3d9-422c-426a-ba34-2a95799f4353 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:50.101663 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914bf3d9-422c-426a-ba34-2a95799f4353 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:50.117841 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1032/2078] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 06 14:59:50.119727 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29893181-2d02-4fb9-b0b1-fd3b14142cd8 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:50.135401 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29893181-2d02-4fb9-b0b1-fd3b14142cd8 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:50.143148 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13916cd6-6981-4fae-98c5-8880e6c9f343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:50.144945 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13916cd6-6981-4fae-98c5-8880e6c9f343 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:50.145620 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13916cd6-6981-4fae-98c5-8880e6c9f343 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:50.145758 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13916cd6-6981-4fae-98c5-8880e6c9f343 admin admin] PATH_INFO: `/v3/projects/4c4671efc7be4313a67a83fbf93234fc/users/88e831bc4ff040f98cb2431f02426312/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:50.173835 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13916cd6-6981-4fae-98c5-8880e6c9f343 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c4671efc7be4313a67a83fbf93234fc, user_id=88e831bc4ff040f98cb2431f02426312, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:50.179792 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13916cd6-6981-4fae-98c5-8880e6c9f343 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:50.201297 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1047/2079] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:59:50 2022] PUT /identity/v3/projects/4c4671efc7be4313a67a83fbf93234fc/users/88e831bc4ff040f98cb2431f02426312/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:50.214257 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173dc2a3-cfa4-4a3a-8e3c-6312864e871a None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:50.215072 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173dc2a3-cfa4-4a3a-8e3c-6312864e871a None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:50.215662 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173dc2a3-cfa4-4a3a-8e3c-6312864e871a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:50.287538 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-173dc2a3-cfa4-4a3a-8e3c-6312864e871a None None] MFA Rules not processed for user `88e831bc4ff040f98cb2431f02426312`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:50.290359 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-173dc2a3-cfa4-4a3a-8e3c-6312864e871a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:50.422365 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1033/2080] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:50 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:50.435278 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13916cd6-6981-4fae-98c5-8880e6c9f343 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:50.440690 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-13916cd6-6981-4fae-98c5-8880e6c9f343 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:50.524672 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13916cd6-6981-4fae-98c5-8880e6c9f343 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:50.529691 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0105e59-b383-4683-9eb3-51e1fdc7e180 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:50.530830 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0105e59-b383-4683-9eb3-51e1fdc7e180 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:50.531281 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0105e59-b383-4683-9eb3-51e1fdc7e180 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:50.531891 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0105e59-b383-4683-9eb3-51e1fdc7e180 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:50.542301 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0105e59-b383-4683-9eb3-51e1fdc7e180 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:50.544165 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0105e59-b383-4683-9eb3-51e1fdc7e180 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:50.557508 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1048/2081] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:59:50 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 14:59:52.151639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23993785-5cd5-4f4c-a1a4-5ccd2b748ba5 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:52.152158 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23993785-5cd5-4f4c-a1a4-5ccd2b748ba5 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:52.152576 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23993785-5cd5-4f4c-a1a4-5ccd2b748ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:52.212203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-23993785-5cd5-4f4c-a1a4-5ccd2b748ba5 None None] MFA Rules not processed for user `4b3af1d82c33446e998f07bce54ef941`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:52.214076 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-23993785-5cd5-4f4c-a1a4-5ccd2b748ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:52.341323 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1034/2082] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:52.342141 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0105e59-b383-4683-9eb3-51e1fdc7e180 service cinder] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:52.343979 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-a0105e59-b383-4683-9eb3-51e1fdc7e180 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:52.424098 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0105e59-b383-4683-9eb3-51e1fdc7e180 service cinder] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:52.429502 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74288d87-bbf3-4724-ab67-973163b48a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:52.430311 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74288d87-bbf3-4724-ab67-973163b48a4f admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:52.430733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74288d87-bbf3-4724-ab67-973163b48a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:52.431076 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74288d87-bbf3-4724-ab67-973163b48a4f admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:52.436991 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74288d87-bbf3-4724-ab67-973163b48a4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:52.439482 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74288d87-bbf3-4724-ab67-973163b48a4f admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:52.474659 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1049/2083] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:59:52 2022] POST /identity/v3/projects => generated 377 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 14:59:52.479261 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23993785-5cd5-4f4c-a1a4-5ccd2b748ba5 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:52.494104 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23993785-5cd5-4f4c-a1a4-5ccd2b748ba5 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:52.573603 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd13339-402e-48b1-8835-93454eea8e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:52.574471 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd13339-402e-48b1-8835-93454eea8e81 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:52.574795 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd13339-402e-48b1-8835-93454eea8e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:52.575320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd13339-402e-48b1-8835-93454eea8e81 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:52.576034 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd13339-402e-48b1-8835-93454eea8e81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:52.578497 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd13339-402e-48b1-8835-93454eea8e81 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:52.613024 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1035/2084] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:59:52 2022] POST /identity/v3/users => generated 295 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:59:52.616814 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74288d87-bbf3-4724-ab67-973163b48a4f admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:52.630696 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74288d87-bbf3-4724-ab67-973163b48a4f admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:52.639839 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec5ae3-b9fd-4af6-baae-db4b1ad130e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:52.640714 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec5ae3-b9fd-4af6-baae-db4b1ad130e4 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:52.641187 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec5ae3-b9fd-4af6-baae-db4b1ad130e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:52.641556 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec5ae3-b9fd-4af6-baae-db4b1ad130e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:52.642272 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec5ae3-b9fd-4af6-baae-db4b1ad130e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:52.644342 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec5ae3-b9fd-4af6-baae-db4b1ad130e4 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:52.701495 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1050/2085] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:59:52.706622 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd13339-402e-48b1-8835-93454eea8e81 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:52.723100 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd13339-402e-48b1-8835-93454eea8e81 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:52.743524 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea7f9b-9803-4af0-ad0f-e189817cb559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:52.744513 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea7f9b-9803-4af0-ad0f-e189817cb559 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:52.744923 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea7f9b-9803-4af0-ad0f-e189817cb559 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:52.745503 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea7f9b-9803-4af0-ad0f-e189817cb559 admin admin] PATH_INFO: `/v3/projects/9835badd92d54c2b9c31ed869e7ba010/users/8959632f4e8e4f61bf569d3fa3c32719/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:52.767436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ea7f9b-9803-4af0-ad0f-e189817cb559 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9835badd92d54c2b9c31ed869e7ba010, user_id=8959632f4e8e4f61bf569d3fa3c32719, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:52.770132 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ea7f9b-9803-4af0-ad0f-e189817cb559 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:52.794455 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1036/2086] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:59:52 2022] PUT /identity/v3/projects/9835badd92d54c2b9c31ed869e7ba010/users/8959632f4e8e4f61bf569d3fa3c32719/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:52.798526 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec5ae3-b9fd-4af6-baae-db4b1ad130e4 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:52.811845 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec5ae3-b9fd-4af6-baae-db4b1ad130e4 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:52.910538 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da51ae7a-0529-4b32-85fd-ce0879fce991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:52.911225 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51ae7a-0529-4b32-85fd-ce0879fce991 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:52.911509 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51ae7a-0529-4b32-85fd-ce0879fce991 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:52.911788 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51ae7a-0529-4b32-85fd-ce0879fce991 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:52.912519 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da51ae7a-0529-4b32-85fd-ce0879fce991 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:52.914435 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da51ae7a-0529-4b32-85fd-ce0879fce991 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:52.925706 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1051/2087] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:59:52.929827 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea7f9b-9803-4af0-ad0f-e189817cb559 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:52.942324 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea7f9b-9803-4af0-ad0f-e189817cb559 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:52.947013 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06ba37-4767-4b24-8f3e-6080842d935c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:52.948263 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06ba37-4767-4b24-8f3e-6080842d935c admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:52.948381 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06ba37-4767-4b24-8f3e-6080842d935c admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:52.948875 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06ba37-4767-4b24-8f3e-6080842d935c admin admin] PATH_INFO: `/v3/projects/9835badd92d54c2b9c31ed869e7ba010/users/8959632f4e8e4f61bf569d3fa3c32719/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:52.950852 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b06ba37-4767-4b24-8f3e-6080842d935c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9835badd92d54c2b9c31ed869e7ba010, user_id=8959632f4e8e4f61bf569d3fa3c32719, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:52.953596 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b06ba37-4767-4b24-8f3e-6080842d935c admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:52.964314 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1037/2088] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:59:52 2022] PUT /identity/v3/projects/9835badd92d54c2b9c31ed869e7ba010/users/8959632f4e8e4f61bf569d3fa3c32719/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 14:59:56.903078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da51ae7a-0529-4b32-85fd-ce0879fce991 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:56.907381 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-da51ae7a-0529-4b32-85fd-ce0879fce991 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:56.938023 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da51ae7a-0529-4b32-85fd-ce0879fce991 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:57.067705 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cef373-5897-4cd0-a689-25bb9dcc5d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:57.068677 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cef373-5897-4cd0-a689-25bb9dcc5d91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:57.074202 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cef373-5897-4cd0-a689-25bb9dcc5d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:57.074477 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cef373-5897-4cd0-a689-25bb9dcc5d91 admin admin] PATH_INFO: `/v3/users/ec6466270f474c149bb6c41dedccc2e5` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:57.076400 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cef373-5897-4cd0-a689-25bb9dcc5d91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec6466270f474c149bb6c41dedccc2e5)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:57.078972 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cef373-5897-4cd0-a689-25bb9dcc5d91 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:57.177588 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b0cef373-5897-4cd0-a689-25bb9dcc5d91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec6466270f474c149bb6c41dedccc2e5'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:57.192946 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b0cef373-5897-4cd0-a689-25bb9dcc5d91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec6466270f474c149bb6c41dedccc2e5'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:57.207574 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b0cef373-5897-4cd0-a689-25bb9dcc5d91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec6466270f474c149bb6c41dedccc2e5'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:57.209696 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b0cef373-5897-4cd0-a689-25bb9dcc5d91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec6466270f474c149bb6c41dedccc2e5'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:57.210351 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b0cef373-5897-4cd0-a689-25bb9dcc5d91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec6466270f474c149bb6c41dedccc2e5'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:57.264060 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1052/2089] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 14:59:56 2022] DELETE /identity/v3/users/ec6466270f474c149bb6c41dedccc2e5 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:57.273712 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657d57c-a7bc-4053-b53a-af0b95817d8e None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:57.274229 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657d57c-a7bc-4053-b53a-af0b95817d8e None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:57.274324 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657d57c-a7bc-4053-b53a-af0b95817d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:57.312696 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-6657d57c-a7bc-4053-b53a-af0b95817d8e None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:57.314365 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-6657d57c-a7bc-4053-b53a-af0b95817d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:57.429037 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1038/2090] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:57.456311 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cef373-5897-4cd0-a689-25bb9dcc5d91 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:57.458149 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-b0cef373-5897-4cd0-a689-25bb9dcc5d91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:57.639495 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cef373-5897-4cd0-a689-25bb9dcc5d91 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:57.658257 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77facc1-97c2-46a7-be1b-0364a1635158 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:57.658939 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77facc1-97c2-46a7-be1b-0364a1635158 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:57.659222 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77facc1-97c2-46a7-be1b-0364a1635158 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:57.659496 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77facc1-97c2-46a7-be1b-0364a1635158 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:57.707503 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77facc1-97c2-46a7-be1b-0364a1635158 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:57.709046 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77facc1-97c2-46a7-be1b-0364a1635158 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:57.749224 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1053/2091] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:59:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 14:59:58.764430 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6657d57c-a7bc-4053-b53a-af0b95817d8e None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:58.765956 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-6657d57c-a7bc-4053-b53a-af0b95817d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:58.787482 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6657d57c-a7bc-4053-b53a-af0b95817d8e None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:58.803864 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5446e2a4-03c4-4983-bfde-7ac59486d982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:58.806699 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446e2a4-03c4-4983-bfde-7ac59486d982 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:58.807270 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446e2a4-03c4-4983-bfde-7ac59486d982 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:58.808151 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446e2a4-03c4-4983-bfde-7ac59486d982 admin admin] PATH_INFO: `/v3/projects/8966ccce76004a49b47282d73d52224f` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:58.811517 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5446e2a4-03c4-4983-bfde-7ac59486d982 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8966ccce76004a49b47282d73d52224f)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:58.816423 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5446e2a4-03c4-4983-bfde-7ac59486d982 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:58.958478 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-5446e2a4-03c4-4983-bfde-7ac59486d982 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8966ccce76004a49b47282d73d52224f', 'request_id': 'req-5446e2a4-03c4-4983-bfde-7ac59486d982'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:58.972938 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-5446e2a4-03c4-4983-bfde-7ac59486d982 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8966ccce76004a49b47282d73d52224f', 'request_id': 'req-5446e2a4-03c4-4983-bfde-7ac59486d982'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:59.011384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-5446e2a4-03c4-4983-bfde-7ac59486d982 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8966ccce76004a49b47282d73d52224f', 'request_id': 'req-5446e2a4-03c4-4983-bfde-7ac59486d982'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:59.037399 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-5446e2a4-03c4-4983-bfde-7ac59486d982 admin admin] The token cache is being invalidate because project 8966ccce76004a49b47282d73d52224f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 14:59:59.037990 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-5446e2a4-03c4-4983-bfde-7ac59486d982 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:59.039917 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-5446e2a4-03c4-4983-bfde-7ac59486d982 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 14:59:59.042665 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1039/2092] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 14:59:58 2022] DELETE /identity/v3/projects/8966ccce76004a49b47282d73d52224f => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:59.047892 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc59ab-894c-47f8-9da3-cbbf734c119b None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:59.048407 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc59ab-894c-47f8-9da3-cbbf734c119b None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:59.048990 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc59ab-894c-47f8-9da3-cbbf734c119b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:59.096237 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-cdcc59ab-894c-47f8-9da3-cbbf734c119b None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:59.098163 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-cdcc59ab-894c-47f8-9da3-cbbf734c119b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:59.267718 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1054/2093] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:59.276496 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b972332-8a85-4c7d-baa8-f349f7fc74b0 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:59.276938 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b972332-8a85-4c7d-baa8-f349f7fc74b0 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:59.277315 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b972332-8a85-4c7d-baa8-f349f7fc74b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:59.311406 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-2b972332-8a85-4c7d-baa8-f349f7fc74b0 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:59.313744 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-2b972332-8a85-4c7d-baa8-f349f7fc74b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:59.329686 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1040/2094] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:59.330938 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcc59ab-894c-47f8-9da3-cbbf734c119b None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:59.345512 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcc59ab-894c-47f8-9da3-cbbf734c119b None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:59.352982 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd5c74-1b20-4c5b-b31b-d1d8f4f86bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:59.354465 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd5c74-1b20-4c5b-b31b-d1d8f4f86bfb admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:59.354921 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd5c74-1b20-4c5b-b31b-d1d8f4f86bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:59.355364 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd5c74-1b20-4c5b-b31b-d1d8f4f86bfb admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:59.356392 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cd5c74-1b20-4c5b-b31b-d1d8f4f86bfb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 14:59:59.357465 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cd5c74-1b20-4c5b-b31b-d1d8f4f86bfb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:59.359621 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cd5c74-1b20-4c5b-b31b-d1d8f4f86bfb admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:59.374170 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1055/2095] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 14:59:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 14:59:59.379318 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b972332-8a85-4c7d-baa8-f349f7fc74b0 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:59.390664 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b972332-8a85-4c7d-baa8-f349f7fc74b0 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:59.396368 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4009842b-e3c3-499c-aa81-70a0aeebf2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:59.397145 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4009842b-e3c3-499c-aa81-70a0aeebf2b4 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:59.397422 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4009842b-e3c3-499c-aa81-70a0aeebf2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:59.397888 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4009842b-e3c3-499c-aa81-70a0aeebf2b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:59.398579 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4009842b-e3c3-499c-aa81-70a0aeebf2b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:59.400524 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4009842b-e3c3-499c-aa81-70a0aeebf2b4 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:59.438660 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1041/2096] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 14:59:59 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 14:59:59.442751 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd5c74-1b20-4c5b-b31b-d1d8f4f86bfb admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:59.455920 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd5c74-1b20-4c5b-b31b-d1d8f4f86bfb admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:59.549065 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72099d3d-389c-43d5-9154-cb690b3395a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:59.550004 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72099d3d-389c-43d5-9154-cb690b3395a4 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:59.550455 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72099d3d-389c-43d5-9154-cb690b3395a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:59.550885 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72099d3d-389c-43d5-9154-cb690b3395a4 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:59.552001 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72099d3d-389c-43d5-9154-cb690b3395a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:59.554686 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72099d3d-389c-43d5-9154-cb690b3395a4 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:59.589336 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1056/2097] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 14:59:59 2022] POST /identity/v3/users => generated 287 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 14:59:59.590068 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4009842b-e3c3-499c-aa81-70a0aeebf2b4 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:59.608956 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4009842b-e3c3-499c-aa81-70a0aeebf2b4 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:59.614035 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39a3a9a-7c5a-4ebc-9ada-dabde5e3f9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:59.614997 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a3a9a-7c5a-4ebc-9ada-dabde5e3f9dd admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:59.615472 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a3a9a-7c5a-4ebc-9ada-dabde5e3f9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:59.615895 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a3a9a-7c5a-4ebc-9ada-dabde5e3f9dd admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:59.616812 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39a3a9a-7c5a-4ebc-9ada-dabde5e3f9dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:59.618595 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39a3a9a-7c5a-4ebc-9ada-dabde5e3f9dd admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:59.634574 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1042/2098] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:59 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 06 14:59:59.635883 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72099d3d-389c-43d5-9154-cb690b3395a4 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:59.647385 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72099d3d-389c-43d5-9154-cb690b3395a4 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:59.652793 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e4e0c-5cde-4d4b-ae7c-7388ac95401b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:59.653655 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e4e0c-5cde-4d4b-ae7c-7388ac95401b admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:59.654117 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e4e0c-5cde-4d4b-ae7c-7388ac95401b admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:59.654399 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e4e0c-5cde-4d4b-ae7c-7388ac95401b admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:59.655123 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e4e0c-5cde-4d4b-ae7c-7388ac95401b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:59.657202 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e4e0c-5cde-4d4b-ae7c-7388ac95401b admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:59.668760 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1057/2099] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 14:59:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 14:59:59.672757 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39a3a9a-7c5a-4ebc-9ada-dabde5e3f9dd admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:59.683562 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39a3a9a-7c5a-4ebc-9ada-dabde5e3f9dd admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 14:59:59.689051 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71541b14-3bb0-45dd-8c60-6219bed418a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 14:59:59.690120 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71541b14-3bb0-45dd-8c60-6219bed418a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:59.690554 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71541b14-3bb0-45dd-8c60-6219bed418a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:59.690999 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71541b14-3bb0-45dd-8c60-6219bed418a0 admin admin] PATH_INFO: `/v3/projects/fb6032585c79402daf0d0846ed336dd5/users/f19eca3041034007b13084b651ebbe77/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:59.709679 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71541b14-3bb0-45dd-8c60-6219bed418a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb6032585c79402daf0d0846ed336dd5, user_id=f19eca3041034007b13084b651ebbe77, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 14:59:59.712540 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71541b14-3bb0-45dd-8c60-6219bed418a0 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 14:59:59.730182 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1043/2100] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 14:59:59 2022] PUT /identity/v3/projects/fb6032585c79402daf0d0846ed336dd5/users/f19eca3041034007b13084b651ebbe77/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 14:59:59.741348 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8f715-d3e4-490a-a85b-be94b8d67c11 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 14:59:59.741656 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8f715-d3e4-490a-a85b-be94b8d67c11 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 14:59:59.741936 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8f715-d3e4-490a-a85b-be94b8d67c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 14:59:59.774327 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-0af8f715-d3e4-490a-a85b-be94b8d67c11 None None] MFA Rules not processed for user `f19eca3041034007b13084b651ebbe77`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 14:59:59.776718 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-0af8f715-d3e4-490a-a85b-be94b8d67c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 14:59:59.957574 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1058/2101] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 14:59:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 14:59:59.972544 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71541b14-3bb0-45dd-8c60-6219bed418a0 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 14:59:59.974356 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-71541b14-3bb0-45dd-8c60-6219bed418a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:00.122375 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71541b14-3bb0-45dd-8c60-6219bed418a0 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:00.127033 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e0051-47d7-42df-b6fa-a5d36ceae3b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:00.127694 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e0051-47d7-42df-b6fa-a5d36ceae3b1 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:00.128184 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e0051-47d7-42df-b6fa-a5d36ceae3b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:00.128650 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e0051-47d7-42df-b6fa-a5d36ceae3b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:00.141349 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600e0051-47d7-42df-b6fa-a5d36ceae3b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:00.143074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600e0051-47d7-42df-b6fa-a5d36ceae3b1 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:00.169073 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1044/2102] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 14:59:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:00:00.972095 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6945e-fd14-46b6-b95b-9e789b810f53 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:00.972585 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6945e-fd14-46b6-b95b-9e789b810f53 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:00.972718 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6945e-fd14-46b6-b95b-9e789b810f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:01.000489 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-1ff6945e-fd14-46b6-b95b-9e789b810f53 None None] MFA Rules not processed for user `8959632f4e8e4f61bf569d3fa3c32719`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:00:01.001718 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-1ff6945e-fd14-46b6-b95b-9e789b810f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:01.069604 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1059/2103] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:00:00 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:00:01.081611 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e0051-47d7-42df-b6fa-a5d36ceae3b1 service cinder] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:01.083089 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-600e0051-47d7-42df-b6fa-a5d36ceae3b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:01.160730 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e0051-47d7-42df-b6fa-a5d36ceae3b1 service cinder] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:01.166141 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c993f75-1607-4129-bef0-84036e6418a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:01.167381 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c993f75-1607-4129-bef0-84036e6418a6 service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:01.167664 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c993f75-1607-4129-bef0-84036e6418a6 service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:01.167939 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c993f75-1607-4129-bef0-84036e6418a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:01.169748 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-9c993f75-1607-4129-bef0-84036e6418a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:01.226160 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c993f75-1607-4129-bef0-84036e6418a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:01.227676 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c993f75-1607-4129-bef0-84036e6418a6 service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:01.237322 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1045/2104] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:00:01 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:00:03.652422 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395490b-a911-4540-80af-c95800dd7283 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:03.652908 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395490b-a911-4540-80af-c95800dd7283 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:03.653137 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395490b-a911-4540-80af-c95800dd7283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:03.674616 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-3395490b-a911-4540-80af-c95800dd7283 None None] MFA Rules not processed for user `fdac43006996421799abf5bd401d0fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:00:03.675849 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-3395490b-a911-4540-80af-c95800dd7283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:03.750683 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1060/2105] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:00:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 15:00:05.518225 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c993f75-1607-4129-bef0-84036e6418a6 service nova] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:05.520172 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-9c993f75-1607-4129-bef0-84036e6418a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:05.637052 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c993f75-1607-4129-bef0-84036e6418a6 service nova] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:05.664563 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecdb82a-840a-49bf-8df2-0a451d0ae021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:05.665872 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecdb82a-840a-49bf-8df2-0a451d0ae021 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:05.666407 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecdb82a-840a-49bf-8df2-0a451d0ae021 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:05.666787 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecdb82a-840a-49bf-8df2-0a451d0ae021 admin admin] PATH_INFO: `/v3/users/a50983a75196418eb3f00328a1a60213` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:05.669024 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecdb82a-840a-49bf-8df2-0a451d0ae021 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a50983a75196418eb3f00328a1a60213)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:05.672198 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecdb82a-840a-49bf-8df2-0a451d0ae021 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:05.753317 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-aecdb82a-840a-49bf-8df2-0a451d0ae021 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a50983a75196418eb3f00328a1a60213'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:05.756369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-aecdb82a-840a-49bf-8df2-0a451d0ae021 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a50983a75196418eb3f00328a1a60213'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:05.782766 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-aecdb82a-840a-49bf-8df2-0a451d0ae021 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a50983a75196418eb3f00328a1a60213'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:05.803132 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-aecdb82a-840a-49bf-8df2-0a451d0ae021 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a50983a75196418eb3f00328a1a60213'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:05.804227 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-aecdb82a-840a-49bf-8df2-0a451d0ae021 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a50983a75196418eb3f00328a1a60213'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:05.850807 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1046/2106] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:00:05 2022] DELETE /identity/v3/users/a50983a75196418eb3f00328a1a60213 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 15:00:06.227185 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3395490b-a911-4540-80af-c95800dd7283 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:06.240280 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3395490b-a911-4540-80af-c95800dd7283 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:06.320221 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a84eb0-cab0-4fb9-b912-341982d2c4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:06.321894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a84eb0-cab0-4fb9-b912-341982d2c4ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:06.322255 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a84eb0-cab0-4fb9-b912-341982d2c4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:06.322611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a84eb0-cab0-4fb9-b912-341982d2c4ee admin admin] PATH_INFO: `/v3/projects/3cab862a707246328df1cfcad6ba10cc` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:06.325292 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a84eb0-cab0-4fb9-b912-341982d2c4ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cab862a707246328df1cfcad6ba10cc)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:06.328100 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a84eb0-cab0-4fb9-b912-341982d2c4ee admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:06.387303 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a3a84eb0-cab0-4fb9-b912-341982d2c4ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cab862a707246328df1cfcad6ba10cc', 'request_id': 'req-a3a84eb0-cab0-4fb9-b912-341982d2c4ee'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:06.393190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a3a84eb0-cab0-4fb9-b912-341982d2c4ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cab862a707246328df1cfcad6ba10cc', 'request_id': 'req-a3a84eb0-cab0-4fb9-b912-341982d2c4ee'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:06.402306 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a3a84eb0-cab0-4fb9-b912-341982d2c4ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cab862a707246328df1cfcad6ba10cc', 'request_id': 'req-a3a84eb0-cab0-4fb9-b912-341982d2c4ee'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:06.416786 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a3a84eb0-cab0-4fb9-b912-341982d2c4ee admin admin] The token cache is being invalidate because project 3cab862a707246328df1cfcad6ba10cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:00:06.416786 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a3a84eb0-cab0-4fb9-b912-341982d2c4ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:06.417515 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a3a84eb0-cab0-4fb9-b912-341982d2c4ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:06.419067 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1061/2107] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:00:06 2022] DELETE /identity/v3/projects/3cab862a707246328df1cfcad6ba10cc => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:00:06.425302 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56934055-d58c-44c9-888a-f2fda4a2c290 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:06.425585 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56934055-d58c-44c9-888a-f2fda4a2c290 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:06.425888 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56934055-d58c-44c9-888a-f2fda4a2c290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:06.446485 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-56934055-d58c-44c9-888a-f2fda4a2c290 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:00:06.448328 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-56934055-d58c-44c9-888a-f2fda4a2c290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:06.543863 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1047/2108] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:00:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:00:06.551986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd944c88-a7ae-4445-94fb-4fcd686fed1d None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:06.552271 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd944c88-a7ae-4445-94fb-4fcd686fed1d None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:06.552551 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd944c88-a7ae-4445-94fb-4fcd686fed1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:06.575103 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-bd944c88-a7ae-4445-94fb-4fcd686fed1d None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:00:06.576468 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-bd944c88-a7ae-4445-94fb-4fcd686fed1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:06.590029 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1062/2109] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:00:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:00:06.594542 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56934055-d58c-44c9-888a-f2fda4a2c290 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:06.605551 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56934055-d58c-44c9-888a-f2fda4a2c290 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:06.610579 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a3726d-1a84-438d-9c64-ed4ca46226f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:06.611536 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a3726d-1a84-438d-9c64-ed4ca46226f5 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:06.612144 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a3726d-1a84-438d-9c64-ed4ca46226f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:06.612426 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a3726d-1a84-438d-9c64-ed4ca46226f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:06.613836 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a3726d-1a84-438d-9c64-ed4ca46226f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 15:00:06.614776 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a3726d-1a84-438d-9c64-ed4ca46226f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:06.616448 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a3726d-1a84-438d-9c64-ed4ca46226f5 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:06.632856 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1048/2110] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 15:00:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 15:00:06.637723 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd944c88-a7ae-4445-94fb-4fcd686fed1d None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:06.649539 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd944c88-a7ae-4445-94fb-4fcd686fed1d None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:06.657325 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567db72-3c35-45c0-a472-62818dd65276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:06.658100 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567db72-3c35-45c0-a472-62818dd65276 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:06.658445 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567db72-3c35-45c0-a472-62818dd65276 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:06.658783 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567db72-3c35-45c0-a472-62818dd65276 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:06.659556 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1567db72-3c35-45c0-a472-62818dd65276 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:06.661597 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1567db72-3c35-45c0-a472-62818dd65276 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:06.694681 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1063/2111] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:00:06 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 15:00:06.695880 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a3726d-1a84-438d-9c64-ed4ca46226f5 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:06.708065 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a3726d-1a84-438d-9c64-ed4ca46226f5 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:06.777558 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7064498-a5a4-4c65-ab18-da99bfbcc7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:06.778246 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7064498-a5a4-4c65-ab18-da99bfbcc7b7 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:06.778533 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7064498-a5a4-4c65-ab18-da99bfbcc7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:06.778815 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7064498-a5a4-4c65-ab18-da99bfbcc7b7 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:06.779512 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7064498-a5a4-4c65-ab18-da99bfbcc7b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:06.781747 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7064498-a5a4-4c65-ab18-da99bfbcc7b7 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:06.805139 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1049/2112] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:00:06 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:00:06.811015 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567db72-3c35-45c0-a472-62818dd65276 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:06.819346 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7064498-a5a4-4c65-ab18-da99bfbcc7b7 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:06.821685 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-c7064498-a5a4-4c65-ab18-da99bfbcc7b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:06.825854 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567db72-3c35-45c0-a472-62818dd65276 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:06.842229 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7064498-a5a4-4c65-ab18-da99bfbcc7b7 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:06.842451 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f29363-b327-4362-8780-a6d9bb355117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:06.843105 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f29363-b327-4362-8780-a6d9bb355117 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:06.843550 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f29363-b327-4362-8780-a6d9bb355117 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:06.843969 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f29363-b327-4362-8780-a6d9bb355117 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:06.844754 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f29363-b327-4362-8780-a6d9bb355117 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:06.846439 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f29363-b327-4362-8780-a6d9bb355117 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:06.847839 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a478fe-3d76-4dc4-8155-d810076ffe78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:06.848830 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a478fe-3d76-4dc4-8155-d810076ffe78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:06.849476 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a478fe-3d76-4dc4-8155-d810076ffe78 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:06.849476 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a478fe-3d76-4dc4-8155-d810076ffe78 admin admin] PATH_INFO: `/v3/users/f19eca3041034007b13084b651ebbe77` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:06.850774 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a478fe-3d76-4dc4-8155-d810076ffe78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f19eca3041034007b13084b651ebbe77)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:06.852932 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a478fe-3d76-4dc4-8155-d810076ffe78 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:06.860093 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1064/2113] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:00:06.864725 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f29363-b327-4362-8780-a6d9bb355117 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:06.877539 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f29363-b327-4362-8780-a6d9bb355117 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:06.883743 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e8fb57-2ccc-46f5-b309-19bae0b385d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:06.884811 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8fb57-2ccc-46f5-b309-19bae0b385d4 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:06.885157 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8fb57-2ccc-46f5-b309-19bae0b385d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:06.885438 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8fb57-2ccc-46f5-b309-19bae0b385d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:06.886839 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e8fb57-2ccc-46f5-b309-19bae0b385d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:06.888282 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e8fb57-2ccc-46f5-b309-19bae0b385d4 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:06.901088 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1065/2114] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 06 15:00:06.905953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e8fb57-2ccc-46f5-b309-19bae0b385d4 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:06.918743 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e8fb57-2ccc-46f5-b309-19bae0b385d4 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:06.925427 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3cc657-1ffa-4797-b883-4e6c14b9e0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:06.926392 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3cc657-1ffa-4797-b883-4e6c14b9e0d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:06.926794 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3cc657-1ffa-4797-b883-4e6c14b9e0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:06.927121 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3cc657-1ffa-4797-b883-4e6c14b9e0d5 admin admin] PATH_INFO: `/v3/projects/85a10400c98145ff9c3a925f53d3a195/users/f16d485a3d8740a0b8d0ea786571de4e/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:06.927349 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e6a478fe-3d76-4dc4-8155-d810076ffe78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f19eca3041034007b13084b651ebbe77'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:06.928105 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e6a478fe-3d76-4dc4-8155-d810076ffe78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f19eca3041034007b13084b651ebbe77'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:06.939630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e6a478fe-3d76-4dc4-8155-d810076ffe78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f19eca3041034007b13084b651ebbe77'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:06.945776 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3cc657-1ffa-4797-b883-4e6c14b9e0d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85a10400c98145ff9c3a925f53d3a195, user_id=f16d485a3d8740a0b8d0ea786571de4e, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:06.949793 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3cc657-1ffa-4797-b883-4e6c14b9e0d5 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:06.951568 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e6a478fe-3d76-4dc4-8155-d810076ffe78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f19eca3041034007b13084b651ebbe77'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:06.952672 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e6a478fe-3d76-4dc4-8155-d810076ffe78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f19eca3041034007b13084b651ebbe77'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:06.984578 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1050/2115] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:00:06 2022] DELETE /identity/v3/users/f19eca3041034007b13084b651ebbe77 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 15:00:06.986854 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1066/2116] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:00:06 2022] PUT /identity/v3/projects/85a10400c98145ff9c3a925f53d3a195/users/f16d485a3d8740a0b8d0ea786571de4e/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:00:06.996561 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ee314-a803-496b-bdae-e9067568db93 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:06.996561 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ee314-a803-496b-bdae-e9067568db93 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:07.001322 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d6993-32b5-49c4-b941-ca8d604e6e22 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:07.001635 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d6993-32b5-49c4-b941-ca8d604e6e22 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:07.002040 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d6993-32b5-49c4-b941-ca8d604e6e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:07.004985 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ee314-a803-496b-bdae-e9067568db93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:07.035422 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-5f9ee314-a803-496b-bdae-e9067568db93 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:00:07.036644 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-5f9ee314-a803-496b-bdae-e9067568db93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:07.059054 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-4b2d6993-32b5-49c4-b941-ca8d604e6e22 None None] MFA Rules not processed for user `f16d485a3d8740a0b8d0ea786571de4e`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:00:07.060473 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-4b2d6993-32b5-49c4-b941-ca8d604e6e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:07.116218 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1051/2117] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:00:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 15:00:07.139958 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9ee314-a803-496b-bdae-e9067568db93 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:07.143083 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-5f9ee314-a803-496b-bdae-e9067568db93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:07.210483 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1067/2118] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:00:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:00:07.225002 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2d6993-32b5-49c4-b941-ca8d604e6e22 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:07.226654 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-4b2d6993-32b5-49c4-b941-ca8d604e6e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:07.272322 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9ee314-a803-496b-bdae-e9067568db93 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:07.281073 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09cb90c-e275-4f15-b0a5-a6b12413af07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:07.281789 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09cb90c-e275-4f15-b0a5-a6b12413af07 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:07.282031 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09cb90c-e275-4f15-b0a5-a6b12413af07 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:07.282266 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09cb90c-e275-4f15-b0a5-a6b12413af07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:07.308015 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09cb90c-e275-4f15-b0a5-a6b12413af07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:07.310981 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09cb90c-e275-4f15-b0a5-a6b12413af07 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:07.325992 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1052/2119] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:00:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:00:07.336513 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2d6993-32b5-49c4-b941-ca8d604e6e22 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:07.344171 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8a6392-ef71-4a35-9c30-1b5166081b10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:07.344974 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8a6392-ef71-4a35-9c30-1b5166081b10 service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:07.345327 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8a6392-ef71-4a35-9c30-1b5166081b10 service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:07.345698 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8a6392-ef71-4a35-9c30-1b5166081b10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:07.355109 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8a6392-ef71-4a35-9c30-1b5166081b10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:07.356821 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8a6392-ef71-4a35-9c30-1b5166081b10 service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:07.372936 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1068/2120] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:00:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:00:07.844648 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09cb90c-e275-4f15-b0a5-a6b12413af07 service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:07.846534 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d09cb90c-e275-4f15-b0a5-a6b12413af07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:07.865989 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09cb90c-e275-4f15-b0a5-a6b12413af07 service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:07.876888 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e12be50-28f3-4fa1-bd9e-8a53065463fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:07.878616 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e12be50-28f3-4fa1-bd9e-8a53065463fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:07.879246 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e12be50-28f3-4fa1-bd9e-8a53065463fc admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:07.880329 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e12be50-28f3-4fa1-bd9e-8a53065463fc admin admin] PATH_INFO: `/v3/projects/fb6032585c79402daf0d0846ed336dd5` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:07.886894 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e12be50-28f3-4fa1-bd9e-8a53065463fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb6032585c79402daf0d0846ed336dd5)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:07.899983 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e12be50-28f3-4fa1-bd9e-8a53065463fc admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:07.983255 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-3e12be50-28f3-4fa1-bd9e-8a53065463fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb6032585c79402daf0d0846ed336dd5', 'request_id': 'req-3e12be50-28f3-4fa1-bd9e-8a53065463fc'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:07.992587 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-3e12be50-28f3-4fa1-bd9e-8a53065463fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb6032585c79402daf0d0846ed336dd5', 'request_id': 'req-3e12be50-28f3-4fa1-bd9e-8a53065463fc'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:08.003789 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-3e12be50-28f3-4fa1-bd9e-8a53065463fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb6032585c79402daf0d0846ed336dd5', 'request_id': 'req-3e12be50-28f3-4fa1-bd9e-8a53065463fc'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:08.025905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-3e12be50-28f3-4fa1-bd9e-8a53065463fc admin admin] The token cache is being invalidate because project fb6032585c79402daf0d0846ed336dd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:00:08.026110 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-3e12be50-28f3-4fa1-bd9e-8a53065463fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:08.034561 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-3e12be50-28f3-4fa1-bd9e-8a53065463fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:08.037010 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1053/2121] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:00:07 2022] DELETE /identity/v3/projects/fb6032585c79402daf0d0846ed336dd5 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:00:08.047586 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663e473-286a-43b9-a652-59e5056f73e1 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:08.049637 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663e473-286a-43b9-a652-59e5056f73e1 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:08.049917 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663e473-286a-43b9-a652-59e5056f73e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:08.099120 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-4663e473-286a-43b9-a652-59e5056f73e1 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:00:08.100552 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-4663e473-286a-43b9-a652-59e5056f73e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:08.193255 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1069/2122] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:00:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:00:08.201759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3bd17-613a-4dca-a109-5575fbf4e04c None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:08.202038 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3bd17-613a-4dca-a109-5575fbf4e04c None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:08.202301 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3bd17-613a-4dca-a109-5575fbf4e04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:08.224740 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-2ff3bd17-613a-4dca-a109-5575fbf4e04c None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:00:08.226206 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-2ff3bd17-613a-4dca-a109-5575fbf4e04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:08.234545 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1054/2123] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:00:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:00:08.239046 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4663e473-286a-43b9-a652-59e5056f73e1 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:08.249818 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4663e473-286a-43b9-a652-59e5056f73e1 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:08.254432 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a1be6-df2a-41d9-924f-16e03a7b0a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:08.255101 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a1be6-df2a-41d9-924f-16e03a7b0a18 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:08.255380 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a1be6-df2a-41d9-924f-16e03a7b0a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:08.255654 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a1be6-df2a-41d9-924f-16e03a7b0a18 admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:08.256381 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170a1be6-df2a-41d9-924f-16e03a7b0a18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 15:00:08.256719 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170a1be6-df2a-41d9-924f-16e03a7b0a18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:08.258733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170a1be6-df2a-41d9-924f-16e03a7b0a18 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:08.279703 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1070/2124] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 15:00:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 15:00:08.282395 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff3bd17-613a-4dca-a109-5575fbf4e04c None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:08.292580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff3bd17-613a-4dca-a109-5575fbf4e04c None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:08.299941 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52289a96-e355-4466-9361-736c58b8efa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:08.301313 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52289a96-e355-4466-9361-736c58b8efa2 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:08.301313 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52289a96-e355-4466-9361-736c58b8efa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:08.301313 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52289a96-e355-4466-9361-736c58b8efa2 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:08.301980 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52289a96-e355-4466-9361-736c58b8efa2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:08.303902 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52289a96-e355-4466-9361-736c58b8efa2 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:08.347358 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1055/2125] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:00:08 2022] POST /identity/v3/projects => generated 377 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:00:08.352082 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a1be6-df2a-41d9-924f-16e03a7b0a18 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:08.366471 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a1be6-df2a-41d9-924f-16e03a7b0a18 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:08.464633 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c4c15a-aa61-43a2-972c-44367e27be34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:08.465565 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c4c15a-aa61-43a2-972c-44367e27be34 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:08.466029 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c4c15a-aa61-43a2-972c-44367e27be34 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:08.466413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c4c15a-aa61-43a2-972c-44367e27be34 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:08.467217 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c4c15a-aa61-43a2-972c-44367e27be34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:08.469308 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c4c15a-aa61-43a2-972c-44367e27be34 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:08.500274 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1071/2126] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:00:08 2022] POST /identity/v3/users => generated 289 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 15:00:08.501558 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52289a96-e355-4466-9361-736c58b8efa2 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:08.513127 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52289a96-e355-4466-9361-736c58b8efa2 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:08.519703 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34e678b-73e1-4e83-b9aa-bbd3d7f8780d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:08.520454 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e678b-73e1-4e83-b9aa-bbd3d7f8780d admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:08.520691 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e678b-73e1-4e83-b9aa-bbd3d7f8780d admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:08.520971 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e678b-73e1-4e83-b9aa-bbd3d7f8780d admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:08.521745 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34e678b-73e1-4e83-b9aa-bbd3d7f8780d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:08.523347 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34e678b-73e1-4e83-b9aa-bbd3d7f8780d admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:08.537600 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1056/2127] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:00:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 06 15:00:08.542088 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c4c15a-aa61-43a2-972c-44367e27be34 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:08.552449 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c4c15a-aa61-43a2-972c-44367e27be34 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:08.557205 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7297eda9-5689-4678-88d6-be02f1a15dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:08.557879 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297eda9-5689-4678-88d6-be02f1a15dae admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:08.558170 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297eda9-5689-4678-88d6-be02f1a15dae admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:08.558576 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297eda9-5689-4678-88d6-be02f1a15dae admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:08.560079 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7297eda9-5689-4678-88d6-be02f1a15dae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:08.561952 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7297eda9-5689-4678-88d6-be02f1a15dae admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:08.576560 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1072/2128] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:00:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:00:08.581182 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34e678b-73e1-4e83-b9aa-bbd3d7f8780d admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:08.591295 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34e678b-73e1-4e83-b9aa-bbd3d7f8780d admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:08.596385 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2144460e-ecf8-40fa-9f6c-012a724c3d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:08.597311 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2144460e-ecf8-40fa-9f6c-012a724c3d61 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:08.597589 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2144460e-ecf8-40fa-9f6c-012a724c3d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:08.598430 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2144460e-ecf8-40fa-9f6c-012a724c3d61 admin admin] PATH_INFO: `/v3/projects/243e195c7e8c4d228c3d33c874184a68/users/ead7f118009b463f971e5f51ccc60ed6/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:08.617018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2144460e-ecf8-40fa-9f6c-012a724c3d61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=243e195c7e8c4d228c3d33c874184a68, user_id=ead7f118009b463f971e5f51ccc60ed6, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:08.620101 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2144460e-ecf8-40fa-9f6c-012a724c3d61 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:08.638866 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1057/2129] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:00:08 2022] PUT /identity/v3/projects/243e195c7e8c4d228c3d33c874184a68/users/ead7f118009b463f971e5f51ccc60ed6/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:00:08.653254 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2623d-b9a0-45fc-9825-326e44dd68e3 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:08.664052 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2623d-b9a0-45fc-9825-326e44dd68e3 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:08.664587 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2623d-b9a0-45fc-9825-326e44dd68e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:08.695492 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-f5b2623d-b9a0-45fc-9825-326e44dd68e3 None None] MFA Rules not processed for user `ead7f118009b463f971e5f51ccc60ed6`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:00:08.696735 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f5b2623d-b9a0-45fc-9825-326e44dd68e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:08.777744 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1073/2130] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:00:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:00:08.783837 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2144460e-ecf8-40fa-9f6c-012a724c3d61 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:08.848807 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2144460e-ecf8-40fa-9f6c-012a724c3d61 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:08.854237 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be1f99-922b-480e-ae31-aa41cb0a1cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:08.854933 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be1f99-922b-480e-ae31-aa41cb0a1cf9 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:08.855220 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be1f99-922b-480e-ae31-aa41cb0a1cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:08.855493 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be1f99-922b-480e-ae31-aa41cb0a1cf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:08.856181 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be1f99-922b-480e-ae31-aa41cb0a1cf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:08.858128 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be1f99-922b-480e-ae31-aa41cb0a1cf9 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:08.884585 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1058/2131] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:00:08 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:00:08.892472 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b2623d-b9a0-45fc-9825-326e44dd68e3 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:08.905031 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b2623d-b9a0-45fc-9825-326e44dd68e3 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:08.968360 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753e05e5-c8d9-4bb6-8643-61e898ca2990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:08.969075 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e05e5-c8d9-4bb6-8643-61e898ca2990 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:08.969358 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e05e5-c8d9-4bb6-8643-61e898ca2990 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:08.969641 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e05e5-c8d9-4bb6-8643-61e898ca2990 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:08.970335 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753e05e5-c8d9-4bb6-8643-61e898ca2990 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:08.972277 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753e05e5-c8d9-4bb6-8643-61e898ca2990 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:08.998655 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1074/2132] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:00:08 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:00:09.007118 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be1f99-922b-480e-ae31-aa41cb0a1cf9 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:09.026099 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be1f99-922b-480e-ae31-aa41cb0a1cf9 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:09.038703 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4bb363-18c5-4471-8b55-728aca555ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:09.039973 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4bb363-18c5-4471-8b55-728aca555ef5 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:09.040562 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4bb363-18c5-4471-8b55-728aca555ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:09.041243 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4bb363-18c5-4471-8b55-728aca555ef5 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:09.042536 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4bb363-18c5-4471-8b55-728aca555ef5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:09.045711 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4bb363-18c5-4471-8b55-728aca555ef5 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:09.059429 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1059/2133] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:00:09 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:00:09.063837 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753e05e5-c8d9-4bb6-8643-61e898ca2990 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:09.075549 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753e05e5-c8d9-4bb6-8643-61e898ca2990 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:09.080236 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7311a7-d480-4a1d-be52-4f0ba116be0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:09.080940 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7311a7-d480-4a1d-be52-4f0ba116be0e admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:09.081217 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7311a7-d480-4a1d-be52-4f0ba116be0e admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:09.081494 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7311a7-d480-4a1d-be52-4f0ba116be0e admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:09.082215 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7311a7-d480-4a1d-be52-4f0ba116be0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:09.083784 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7311a7-d480-4a1d-be52-4f0ba116be0e admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:09.094393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1075/2134] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:00:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 06 15:00:09.098891 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4bb363-18c5-4471-8b55-728aca555ef5 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:09.109343 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4bb363-18c5-4471-8b55-728aca555ef5 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:09.114263 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc591c-b849-4b85-ab2c-1d00332e9420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:09.115208 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc591c-b849-4b85-ab2c-1d00332e9420 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:09.115504 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc591c-b849-4b85-ab2c-1d00332e9420 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:09.115780 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc591c-b849-4b85-ab2c-1d00332e9420 admin admin] PATH_INFO: `/v3/projects/f0c3a529ee86480b80fcc17ded77a46f/users/f4a7f5c4263e4faab215739da9593687/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:09.135494 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc591c-b849-4b85-ab2c-1d00332e9420 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0c3a529ee86480b80fcc17ded77a46f, user_id=f4a7f5c4263e4faab215739da9593687, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:09.138317 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc591c-b849-4b85-ab2c-1d00332e9420 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:09.162039 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1060/2135] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:00:09 2022] PUT /identity/v3/projects/f0c3a529ee86480b80fcc17ded77a46f/users/f4a7f5c4263e4faab215739da9593687/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 15:00:09.173032 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe503edf-57ca-4a62-974a-182f1832c367 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:09.173362 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe503edf-57ca-4a62-974a-182f1832c367 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:09.174672 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe503edf-57ca-4a62-974a-182f1832c367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:09.203659 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-fe503edf-57ca-4a62-974a-182f1832c367 None None] MFA Rules not processed for user `f4a7f5c4263e4faab215739da9593687`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:00:09.204909 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-fe503edf-57ca-4a62-974a-182f1832c367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:09.344929 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1076/2136] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:00:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 15:00:09.349136 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc591c-b849-4b85-ab2c-1d00332e9420 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:09.427955 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc591c-b849-4b85-ab2c-1d00332e9420 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:09.437606 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee82ab14-b020-4b62-ae93-7987caa785f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:09.438284 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82ab14-b020-4b62-ae93-7987caa785f3 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:09.438571 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82ab14-b020-4b62-ae93-7987caa785f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:09.438850 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82ab14-b020-4b62-ae93-7987caa785f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:09.442640 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee82ab14-b020-4b62-ae93-7987caa785f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:09.442640 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee82ab14-b020-4b62-ae93-7987caa785f3 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:09.472416 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1061/2137] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:00:09 2022] POST /identity/v3/projects => generated 377 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:00:09.477008 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe503edf-57ca-4a62-974a-182f1832c367 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:09.490783 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe503edf-57ca-4a62-974a-182f1832c367 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:09.569844 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b6cbb2-b16e-4e6b-b970-1b5bb185f750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:09.570806 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b6cbb2-b16e-4e6b-b970-1b5bb185f750 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:09.571117 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b6cbb2-b16e-4e6b-b970-1b5bb185f750 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:09.571464 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b6cbb2-b16e-4e6b-b970-1b5bb185f750 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:09.572874 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b6cbb2-b16e-4e6b-b970-1b5bb185f750 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:09.575076 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b6cbb2-b16e-4e6b-b970-1b5bb185f750 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:09.602345 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1077/2138] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:00:09 2022] POST /identity/v3/users => generated 295 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:00:09.607074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee82ab14-b020-4b62-ae93-7987caa785f3 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:09.618742 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee82ab14-b020-4b62-ae93-7987caa785f3 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:09.625276 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f37e3-a2a7-4883-a94e-fa8df06dd914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:09.626130 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f37e3-a2a7-4883-a94e-fa8df06dd914 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:09.627151 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f37e3-a2a7-4883-a94e-fa8df06dd914 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:09.627696 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f37e3-a2a7-4883-a94e-fa8df06dd914 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:09.628536 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9f37e3-a2a7-4883-a94e-fa8df06dd914 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:09.630246 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9f37e3-a2a7-4883-a94e-fa8df06dd914 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:09.649064 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1062/2139] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:00:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:00:09.653117 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b6cbb2-b16e-4e6b-b970-1b5bb185f750 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:09.663797 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b6cbb2-b16e-4e6b-b970-1b5bb185f750 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:09.670153 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e1062-3067-4ad7-9d05-8d9679e1f368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:09.670888 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e1062-3067-4ad7-9d05-8d9679e1f368 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:09.671172 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e1062-3067-4ad7-9d05-8d9679e1f368 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:09.671450 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e1062-3067-4ad7-9d05-8d9679e1f368 admin admin] PATH_INFO: `/v3/projects/d0e7efcbf5cb43f5abbe229fd73bbc25/users/257a348dd3b94e98972202bdabfbced0/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:09.688484 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e1062-3067-4ad7-9d05-8d9679e1f368 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0e7efcbf5cb43f5abbe229fd73bbc25, user_id=257a348dd3b94e98972202bdabfbced0, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:09.696685 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e1062-3067-4ad7-9d05-8d9679e1f368 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:09.718252 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1078/2140] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:00:09 2022] PUT /identity/v3/projects/d0e7efcbf5cb43f5abbe229fd73bbc25/users/257a348dd3b94e98972202bdabfbced0/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:00:09.722297 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f37e3-a2a7-4883-a94e-fa8df06dd914 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:09.740012 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f37e3-a2a7-4883-a94e-fa8df06dd914 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:09.817286 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0439b3-9c45-499b-8b79-5fdc0e2fbf67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:09.818061 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0439b3-9c45-499b-8b79-5fdc0e2fbf67 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:09.818420 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0439b3-9c45-499b-8b79-5fdc0e2fbf67 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:09.818772 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0439b3-9c45-499b-8b79-5fdc0e2fbf67 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:09.819570 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0439b3-9c45-499b-8b79-5fdc0e2fbf67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:09.821304 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0439b3-9c45-499b-8b79-5fdc0e2fbf67 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:09.831583 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1063/2141] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:00:09 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:00:09.835805 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e1062-3067-4ad7-9d05-8d9679e1f368 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:09.847067 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e1062-3067-4ad7-9d05-8d9679e1f368 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:09.853309 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff3120c-cd03-48c9-97b3-9675090daad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:09.855010 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3120c-cd03-48c9-97b3-9675090daad5 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:09.855340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3120c-cd03-48c9-97b3-9675090daad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:09.855738 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3120c-cd03-48c9-97b3-9675090daad5 admin admin] PATH_INFO: `/v3/projects/d0e7efcbf5cb43f5abbe229fd73bbc25/users/257a348dd3b94e98972202bdabfbced0/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:09.858055 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff3120c-cd03-48c9-97b3-9675090daad5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0e7efcbf5cb43f5abbe229fd73bbc25, user_id=257a348dd3b94e98972202bdabfbced0, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:09.861223 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff3120c-cd03-48c9-97b3-9675090daad5 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:09.874205 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1079/2142] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:00:09 2022] PUT /identity/v3/projects/d0e7efcbf5cb43f5abbe229fd73bbc25/users/257a348dd3b94e98972202bdabfbced0/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:00:09.885186 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb667c-e7f7-4ee8-8a35-0364e410e456 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:09.885739 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb667c-e7f7-4ee8-8a35-0364e410e456 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:09.885739 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb667c-e7f7-4ee8-8a35-0364e410e456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:09.914792 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-b0bb667c-e7f7-4ee8-8a35-0364e410e456 None None] MFA Rules not processed for user `257a348dd3b94e98972202bdabfbced0`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:00:09.916639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b0bb667c-e7f7-4ee8-8a35-0364e410e456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:10.033021 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1064/2143] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:00:09 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:00:10.045896 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff3120c-cd03-48c9-97b3-9675090daad5 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:10.049425 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-7ff3120c-cd03-48c9-97b3-9675090daad5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:10.112813 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff3120c-cd03-48c9-97b3-9675090daad5 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:10.116676 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c78a9a-ebd8-485a-a042-f8e4bb3d4347 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:10.117374 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c78a9a-ebd8-485a-a042-f8e4bb3d4347 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:10.117835 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c78a9a-ebd8-485a-a042-f8e4bb3d4347 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:10.118117 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c78a9a-ebd8-485a-a042-f8e4bb3d4347 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:10.180094 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c78a9a-ebd8-485a-a042-f8e4bb3d4347 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:10.182084 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c78a9a-ebd8-485a-a042-f8e4bb3d4347 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:10.191301 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1080/2144] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:00:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:00:14.913754 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb667c-e7f7-4ee8-8a35-0364e410e456 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:14.935186 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb667c-e7f7-4ee8-8a35-0364e410e456 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:14.939089 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73303bb7-0d15-436a-ae41-5b0098dcf6f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:14.939752 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73303bb7-0d15-436a-ae41-5b0098dcf6f4 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:14.940040 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73303bb7-0d15-436a-ae41-5b0098dcf6f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:14.940326 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73303bb7-0d15-436a-ae41-5b0098dcf6f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:15.036145 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73303bb7-0d15-436a-ae41-5b0098dcf6f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:15.037908 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73303bb7-0d15-436a-ae41-5b0098dcf6f4 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:15.048323 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1065/2145] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:00:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:00:15.763682 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c78a9a-ebd8-485a-a042-f8e4bb3d4347 service cinder] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:15.793484 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c78a9a-ebd8-485a-a042-f8e4bb3d4347 service cinder] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:15.793484 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc0f6e3-2b3c-4019-b756-0d12acd95e53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:15.793484 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0f6e3-2b3c-4019-b756-0d12acd95e53 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:15.793868 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0f6e3-2b3c-4019-b756-0d12acd95e53 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:15.793980 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0f6e3-2b3c-4019-b756-0d12acd95e53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:15.815635 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc0f6e3-2b3c-4019-b756-0d12acd95e53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:15.817341 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc0f6e3-2b3c-4019-b756-0d12acd95e53 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:15.828845 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1081/2146] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:00:15 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:00:17.093342 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73303bb7-0d15-436a-ae41-5b0098dcf6f4 service cinder] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:17.094786 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-73303bb7-0d15-436a-ae41-5b0098dcf6f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:17.244384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73303bb7-0d15-436a-ae41-5b0098dcf6f4 service cinder] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:17.249565 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f7714d-356e-4d3b-ab7c-00159af94006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:17.250631 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7714d-356e-4d3b-ab7c-00159af94006 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:17.250916 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7714d-356e-4d3b-ab7c-00159af94006 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:17.251182 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7714d-356e-4d3b-ab7c-00159af94006 admin admin] PATH_INFO: `/v3/users/f16d485a3d8740a0b8d0ea786571de4e` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:17.252775 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f7714d-356e-4d3b-ab7c-00159af94006 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f16d485a3d8740a0b8d0ea786571de4e)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:17.256885 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f7714d-356e-4d3b-ab7c-00159af94006 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:17.329466 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-89f7714d-356e-4d3b-ab7c-00159af94006 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f16d485a3d8740a0b8d0ea786571de4e'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:17.330191 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-89f7714d-356e-4d3b-ab7c-00159af94006 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f16d485a3d8740a0b8d0ea786571de4e'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:17.346574 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-89f7714d-356e-4d3b-ab7c-00159af94006 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f16d485a3d8740a0b8d0ea786571de4e'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:17.366799 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-89f7714d-356e-4d3b-ab7c-00159af94006 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f16d485a3d8740a0b8d0ea786571de4e'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:17.367699 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-89f7714d-356e-4d3b-ab7c-00159af94006 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f16d485a3d8740a0b8d0ea786571de4e'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:17.396708 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1066/2147] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:00:17 2022] DELETE /identity/v3/users/f16d485a3d8740a0b8d0ea786571de4e => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:00:17.397415 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70476e24-f7ea-4156-9667-248fb1c7931b None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:17.397862 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70476e24-f7ea-4156-9667-248fb1c7931b None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:17.398232 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70476e24-f7ea-4156-9667-248fb1c7931b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:17.429950 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-70476e24-f7ea-4156-9667-248fb1c7931b None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:00:17.431275 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-70476e24-f7ea-4156-9667-248fb1c7931b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:17.512091 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1082/2148] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:00:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:00:17.525943 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f7714d-356e-4d3b-ab7c-00159af94006 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:17.529100 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-89f7714d-356e-4d3b-ab7c-00159af94006 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:17.660823 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f7714d-356e-4d3b-ab7c-00159af94006 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:17.664792 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ba72f5-4dba-4975-9f79-fa2891bb2578 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:17.665452 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba72f5-4dba-4975-9f79-fa2891bb2578 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:17.665731 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba72f5-4dba-4975-9f79-fa2891bb2578 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:17.666001 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba72f5-4dba-4975-9f79-fa2891bb2578 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:17.676749 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ba72f5-4dba-4975-9f79-fa2891bb2578 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:17.678206 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ba72f5-4dba-4975-9f79-fa2891bb2578 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:17.692849 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1067/2149] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:00:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:00:18.108415 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70476e24-f7ea-4156-9667-248fb1c7931b None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:18.111159 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-70476e24-f7ea-4156-9667-248fb1c7931b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:18.130869 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70476e24-f7ea-4156-9667-248fb1c7931b None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:18.139177 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d247e512-364f-46e1-9a86-3d1502e509a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:18.140652 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d247e512-364f-46e1-9a86-3d1502e509a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:18.141241 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d247e512-364f-46e1-9a86-3d1502e509a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:18.141747 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d247e512-364f-46e1-9a86-3d1502e509a9 admin admin] PATH_INFO: `/v3/projects/85a10400c98145ff9c3a925f53d3a195` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:18.143305 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d247e512-364f-46e1-9a86-3d1502e509a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85a10400c98145ff9c3a925f53d3a195)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:18.145897 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d247e512-364f-46e1-9a86-3d1502e509a9 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:18.211717 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d247e512-364f-46e1-9a86-3d1502e509a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85a10400c98145ff9c3a925f53d3a195', 'request_id': 'req-d247e512-364f-46e1-9a86-3d1502e509a9'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:18.218305 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d247e512-364f-46e1-9a86-3d1502e509a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85a10400c98145ff9c3a925f53d3a195', 'request_id': 'req-d247e512-364f-46e1-9a86-3d1502e509a9'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:18.226284 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d247e512-364f-46e1-9a86-3d1502e509a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85a10400c98145ff9c3a925f53d3a195', 'request_id': 'req-d247e512-364f-46e1-9a86-3d1502e509a9'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:18.238588 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d247e512-364f-46e1-9a86-3d1502e509a9 admin admin] The token cache is being invalidate because project 85a10400c98145ff9c3a925f53d3a195 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:00:18.238874 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d247e512-364f-46e1-9a86-3d1502e509a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:18.239505 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d247e512-364f-46e1-9a86-3d1502e509a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:18.241137 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1083/2150] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:00:18 2022] DELETE /identity/v3/projects/85a10400c98145ff9c3a925f53d3a195 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:00:18.249944 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d466c-aec1-43cc-ac85-5829353c901e None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:18.250436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d466c-aec1-43cc-ac85-5829353c901e None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:18.250436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d466c-aec1-43cc-ac85-5829353c901e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:18.288509 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-2c9d466c-aec1-43cc-ac85-5829353c901e None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:00:18.289890 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-2c9d466c-aec1-43cc-ac85-5829353c901e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:18.364770 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1068/2151] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:00:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:00:18.367970 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c59c61-f43e-43ab-885e-8707ddd5a6f7 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:18.368260 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c59c61-f43e-43ab-885e-8707ddd5a6f7 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:18.368534 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c59c61-f43e-43ab-885e-8707ddd5a6f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:18.390728 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-a7c59c61-f43e-43ab-885e-8707ddd5a6f7 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:00:18.391958 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-a7c59c61-f43e-43ab-885e-8707ddd5a6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:18.403052 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1084/2152] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:00:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:00:18.407998 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9d466c-aec1-43cc-ac85-5829353c901e None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:18.419276 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9d466c-aec1-43cc-ac85-5829353c901e None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:18.424658 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a71f438-8958-47ff-912b-bb5cc40bf62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:18.428359 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a71f438-8958-47ff-912b-bb5cc40bf62e admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:18.428359 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a71f438-8958-47ff-912b-bb5cc40bf62e admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:18.428359 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a71f438-8958-47ff-912b-bb5cc40bf62e admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:18.428359 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a71f438-8958-47ff-912b-bb5cc40bf62e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 15:00:18.428359 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a71f438-8958-47ff-912b-bb5cc40bf62e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:18.428359 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a71f438-8958-47ff-912b-bb5cc40bf62e admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:18.442981 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1069/2153] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 15:00:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 15:00:18.447176 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c59c61-f43e-43ab-885e-8707ddd5a6f7 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:18.456332 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c59c61-f43e-43ab-885e-8707ddd5a6f7 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:18.461771 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e749a8-6f36-4dc7-9f88-f80aafc20bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:18.462438 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e749a8-6f36-4dc7-9f88-f80aafc20bf0 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:18.462778 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e749a8-6f36-4dc7-9f88-f80aafc20bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:18.462983 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e749a8-6f36-4dc7-9f88-f80aafc20bf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:18.464084 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e749a8-6f36-4dc7-9f88-f80aafc20bf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:18.466037 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e749a8-6f36-4dc7-9f88-f80aafc20bf0 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:18.498456 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1085/2154] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:00:18 2022] POST /identity/v3/projects => generated 367 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:00:18.502625 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a71f438-8958-47ff-912b-bb5cc40bf62e admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:18.514185 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a71f438-8958-47ff-912b-bb5cc40bf62e admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:18.589436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6f294-6814-47bc-9cc6-7f25da91f897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:18.590043 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6f294-6814-47bc-9cc6-7f25da91f897 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:18.590352 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6f294-6814-47bc-9cc6-7f25da91f897 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:18.590664 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6f294-6814-47bc-9cc6-7f25da91f897 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:18.591333 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b6f294-6814-47bc-9cc6-7f25da91f897 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:18.593293 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b6f294-6814-47bc-9cc6-7f25da91f897 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:18.627274 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1070/2155] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:00:18 2022] POST /identity/v3/users => generated 284 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:00:18.632353 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e749a8-6f36-4dc7-9f88-f80aafc20bf0 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:18.644041 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e749a8-6f36-4dc7-9f88-f80aafc20bf0 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:18.648562 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dbfd00-01ef-44c0-8529-f1e5e8805011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:18.649308 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbfd00-01ef-44c0-8529-f1e5e8805011 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:18.649944 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbfd00-01ef-44c0-8529-f1e5e8805011 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:18.649944 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbfd00-01ef-44c0-8529-f1e5e8805011 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:18.650550 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dbfd00-01ef-44c0-8529-f1e5e8805011 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:18.652735 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dbfd00-01ef-44c0-8529-f1e5e8805011 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:18.665804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1086/2156] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:00:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:00:18.670655 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6f294-6814-47bc-9cc6-7f25da91f897 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:18.682247 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6f294-6814-47bc-9cc6-7f25da91f897 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:18.687319 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ca267-13ca-4c2b-9be8-9862875f5b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:18.687998 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ca267-13ca-4c2b-9be8-9862875f5b4c admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:18.688304 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ca267-13ca-4c2b-9be8-9862875f5b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:18.688556 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ca267-13ca-4c2b-9be8-9862875f5b4c admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:18.689327 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027ca267-13ca-4c2b-9be8-9862875f5b4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:18.690941 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027ca267-13ca-4c2b-9be8-9862875f5b4c admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:18.705941 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1071/2157] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:00:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:00:18.707052 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dbfd00-01ef-44c0-8529-f1e5e8805011 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:18.719460 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dbfd00-01ef-44c0-8529-f1e5e8805011 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:18.724525 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316cbc75-c57e-4b8a-ab26-2c45050328e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:18.725465 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316cbc75-c57e-4b8a-ab26-2c45050328e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:18.725874 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316cbc75-c57e-4b8a-ab26-2c45050328e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:18.726210 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316cbc75-c57e-4b8a-ab26-2c45050328e5 admin admin] PATH_INFO: `/v3/projects/e6ccaaa7c7734c6299a0afe65d1fdabb/users/0ce5ddf308894e4faf16b8d186151371/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:18.744081 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316cbc75-c57e-4b8a-ab26-2c45050328e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6ccaaa7c7734c6299a0afe65d1fdabb, user_id=0ce5ddf308894e4faf16b8d186151371, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:18.746822 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316cbc75-c57e-4b8a-ab26-2c45050328e5 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:18.762917 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1087/2158] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:00:18 2022] PUT /identity/v3/projects/e6ccaaa7c7734c6299a0afe65d1fdabb/users/0ce5ddf308894e4faf16b8d186151371/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:00:18.774024 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a582df5-ce92-4c57-8c66-2fa3528f5d19 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:18.774024 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a582df5-ce92-4c57-8c66-2fa3528f5d19 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:18.774024 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a582df5-ce92-4c57-8c66-2fa3528f5d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:18.804204 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-9a582df5-ce92-4c57-8c66-2fa3528f5d19 None None] MFA Rules not processed for user `0ce5ddf308894e4faf16b8d186151371`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:00:18.806421 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-9a582df5-ce92-4c57-8c66-2fa3528f5d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:18.887581 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1072/2159] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:00:18 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:00:18.892210 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316cbc75-c57e-4b8a-ab26-2c45050328e5 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:18.961403 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316cbc75-c57e-4b8a-ab26-2c45050328e5 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:18.966302 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e45881-16f1-4ddb-8711-d74742b69c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:18.966980 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e45881-16f1-4ddb-8711-d74742b69c34 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:18.967264 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e45881-16f1-4ddb-8711-d74742b69c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:18.967540 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e45881-16f1-4ddb-8711-d74742b69c34 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:18.968262 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e45881-16f1-4ddb-8711-d74742b69c34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:18.970193 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e45881-16f1-4ddb-8711-d74742b69c34 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:19.005526 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1088/2160] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:00:18 2022] POST /identity/v3/projects => generated 369 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:00:19.007904 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a582df5-ce92-4c57-8c66-2fa3528f5d19 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:19.019341 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a582df5-ce92-4c57-8c66-2fa3528f5d19 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:19.092948 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828c8a14-90f5-415c-909b-12c2db2bec9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:19.093872 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828c8a14-90f5-415c-909b-12c2db2bec9e admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:19.094128 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828c8a14-90f5-415c-909b-12c2db2bec9e admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:19.094408 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828c8a14-90f5-415c-909b-12c2db2bec9e admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:19.095238 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828c8a14-90f5-415c-909b-12c2db2bec9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:19.097229 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828c8a14-90f5-415c-909b-12c2db2bec9e admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:19.121999 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1073/2161] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:00:19 2022] POST /identity/v3/users => generated 291 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:00:19.126666 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e45881-16f1-4ddb-8711-d74742b69c34 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:19.138967 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e45881-16f1-4ddb-8711-d74742b69c34 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:19.144439 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270767c-a199-4090-b018-0e84ac10057e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:19.145442 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270767c-a199-4090-b018-0e84ac10057e admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:19.145838 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270767c-a199-4090-b018-0e84ac10057e admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:19.146218 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270767c-a199-4090-b018-0e84ac10057e admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:19.147020 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f270767c-a199-4090-b018-0e84ac10057e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:19.148726 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f270767c-a199-4090-b018-0e84ac10057e admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:19.173422 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1089/2162] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:00:19 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:00:19.180582 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828c8a14-90f5-415c-909b-12c2db2bec9e admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:19.190863 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828c8a14-90f5-415c-909b-12c2db2bec9e admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:19.197016 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b842fa48-bc0e-4768-85ec-fe465f96eefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:19.198513 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b842fa48-bc0e-4768-85ec-fe465f96eefc admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:19.198915 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b842fa48-bc0e-4768-85ec-fe465f96eefc admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:19.199139 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b842fa48-bc0e-4768-85ec-fe465f96eefc admin admin] PATH_INFO: `/v3/projects/07fe248917ae4b468a011e204d9b9003/users/3dcef02cb0b04fc18f821bbec9f3a307/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:19.216453 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b842fa48-bc0e-4768-85ec-fe465f96eefc admin admin] RBAC: Authorizing `identity:create_grant(project_id=07fe248917ae4b468a011e204d9b9003, user_id=3dcef02cb0b04fc18f821bbec9f3a307, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:19.219776 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b842fa48-bc0e-4768-85ec-fe465f96eefc admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:19.234985 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1074/2163] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:00:19 2022] PUT /identity/v3/projects/07fe248917ae4b468a011e204d9b9003/users/3dcef02cb0b04fc18f821bbec9f3a307/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 15:00:19.238980 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270767c-a199-4090-b018-0e84ac10057e admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:19.251350 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270767c-a199-4090-b018-0e84ac10057e admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:19.324279 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733e4206-9dae-4763-a0ef-04fcfb8719b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:19.324892 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e4206-9dae-4763-a0ef-04fcfb8719b0 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:19.325282 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e4206-9dae-4763-a0ef-04fcfb8719b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:19.325561 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e4206-9dae-4763-a0ef-04fcfb8719b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:19.326283 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733e4206-9dae-4763-a0ef-04fcfb8719b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:19.327872 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733e4206-9dae-4763-a0ef-04fcfb8719b0 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:19.338251 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1090/2164] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:00:19 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:00:19.343418 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b842fa48-bc0e-4768-85ec-fe465f96eefc admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:19.354051 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b842fa48-bc0e-4768-85ec-fe465f96eefc admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:19.358708 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529a6935-9b25-400d-a127-5086a7b99ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:19.359597 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529a6935-9b25-400d-a127-5086a7b99ac7 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:19.359876 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529a6935-9b25-400d-a127-5086a7b99ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:19.360149 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529a6935-9b25-400d-a127-5086a7b99ac7 admin admin] PATH_INFO: `/v3/projects/07fe248917ae4b468a011e204d9b9003/users/3dcef02cb0b04fc18f821bbec9f3a307/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:19.362157 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529a6935-9b25-400d-a127-5086a7b99ac7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07fe248917ae4b468a011e204d9b9003, user_id=3dcef02cb0b04fc18f821bbec9f3a307, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:19.364807 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529a6935-9b25-400d-a127-5086a7b99ac7 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:19.376224 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1075/2165] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:00:19 2022] PUT /identity/v3/projects/07fe248917ae4b468a011e204d9b9003/users/3dcef02cb0b04fc18f821bbec9f3a307/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:00:19.388901 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3f975-19b9-4b78-a002-3d2e11082c28 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:19.389182 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3f975-19b9-4b78-a002-3d2e11082c28 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:19.389468 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3f975-19b9-4b78-a002-3d2e11082c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:19.418035 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-7cd3f975-19b9-4b78-a002-3d2e11082c28 None None] MFA Rules not processed for user `3dcef02cb0b04fc18f821bbec9f3a307`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:00:19.419242 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-7cd3f975-19b9-4b78-a002-3d2e11082c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:19.507086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1091/2166] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:00:19 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:00:19.523372 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529a6935-9b25-400d-a127-5086a7b99ac7 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:19.525910 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-529a6935-9b25-400d-a127-5086a7b99ac7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:19.593211 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529a6935-9b25-400d-a127-5086a7b99ac7 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:19.596828 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f73e666-eff9-482f-a6f8-f3d6dfb31eb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:19.597473 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73e666-eff9-482f-a6f8-f3d6dfb31eb9 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:19.597748 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73e666-eff9-482f-a6f8-f3d6dfb31eb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:19.598018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73e666-eff9-482f-a6f8-f3d6dfb31eb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:19.607988 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f73e666-eff9-482f-a6f8-f3d6dfb31eb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:19.609452 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f73e666-eff9-482f-a6f8-f3d6dfb31eb9 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:19.621379 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1076/2167] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:00:19 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:00:19.672384 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd3f975-19b9-4b78-a002-3d2e11082c28 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:19.673852 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-7cd3f975-19b9-4b78-a002-3d2e11082c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:19.687762 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd3f975-19b9-4b78-a002-3d2e11082c28 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:19.691527 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15af08-d31d-47fd-8bbc-040f5427d9dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:19.692189 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15af08-d31d-47fd-8bbc-040f5427d9dc service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:19.692474 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15af08-d31d-47fd-8bbc-040f5427d9dc service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:19.692802 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15af08-d31d-47fd-8bbc-040f5427d9dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:19.760167 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a15af08-d31d-47fd-8bbc-040f5427d9dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:19.764410 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a15af08-d31d-47fd-8bbc-040f5427d9dc service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:19.784696 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1092/2168] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:00:19 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:00:19.846601 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f73e666-eff9-482f-a6f8-f3d6dfb31eb9 service cinder] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:19.848217 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-2f73e666-eff9-482f-a6f8-f3d6dfb31eb9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:19.933790 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f73e666-eff9-482f-a6f8-f3d6dfb31eb9 service cinder] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:19.939536 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254801fa-4579-404c-9374-ec8c9b51aabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:19.940619 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254801fa-4579-404c-9374-ec8c9b51aabd admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:19.940998 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254801fa-4579-404c-9374-ec8c9b51aabd admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:19.941507 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254801fa-4579-404c-9374-ec8c9b51aabd admin admin] PATH_INFO: `/v3/users/ead7f118009b463f971e5f51ccc60ed6` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:19.943931 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254801fa-4579-404c-9374-ec8c9b51aabd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ead7f118009b463f971e5f51ccc60ed6)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:19.946902 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254801fa-4579-404c-9374-ec8c9b51aabd admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:20.033901 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-254801fa-4579-404c-9374-ec8c9b51aabd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ead7f118009b463f971e5f51ccc60ed6'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:20.035234 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-254801fa-4579-404c-9374-ec8c9b51aabd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ead7f118009b463f971e5f51ccc60ed6'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:20.052466 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-254801fa-4579-404c-9374-ec8c9b51aabd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ead7f118009b463f971e5f51ccc60ed6'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:20.071725 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-254801fa-4579-404c-9374-ec8c9b51aabd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ead7f118009b463f971e5f51ccc60ed6'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:20.072481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-254801fa-4579-404c-9374-ec8c9b51aabd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ead7f118009b463f971e5f51ccc60ed6'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:20.095080 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1077/2169] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:00:19 2022] DELETE /identity/v3/users/ead7f118009b463f971e5f51ccc60ed6 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:00:20.100087 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81a596-4617-491a-aca6-16123945fe71 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:20.100415 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81a596-4617-491a-aca6-16123945fe71 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:20.100826 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81a596-4617-491a-aca6-16123945fe71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:20.124303 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-0e81a596-4617-491a-aca6-16123945fe71 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:00:20.127460 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-0e81a596-4617-491a-aca6-16123945fe71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:20.206872 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1093/2170] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:00:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:00:20.216729 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254801fa-4579-404c-9374-ec8c9b51aabd admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:20.218404 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-254801fa-4579-404c-9374-ec8c9b51aabd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:20.284786 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254801fa-4579-404c-9374-ec8c9b51aabd admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:20.289970 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1db0678-4f3a-4a71-8f31-25abbacdca2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:20.290680 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1db0678-4f3a-4a71-8f31-25abbacdca2a service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:20.291121 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1db0678-4f3a-4a71-8f31-25abbacdca2a service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:20.291252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1db0678-4f3a-4a71-8f31-25abbacdca2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:20.306568 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1db0678-4f3a-4a71-8f31-25abbacdca2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:20.309492 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1db0678-4f3a-4a71-8f31-25abbacdca2a service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:20.322500 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1078/2171] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:00:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:00:20.668665 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e81a596-4617-491a-aca6-16123945fe71 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:20.670136 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-0e81a596-4617-491a-aca6-16123945fe71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:20.684246 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e81a596-4617-491a-aca6-16123945fe71 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:20.689109 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59075c5-1ae4-4e59-8785-01acd7dc5e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:20.689982 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59075c5-1ae4-4e59-8785-01acd7dc5e08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:20.690263 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59075c5-1ae4-4e59-8785-01acd7dc5e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:20.695340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59075c5-1ae4-4e59-8785-01acd7dc5e08 admin admin] PATH_INFO: `/v3/projects/243e195c7e8c4d228c3d33c874184a68` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:20.698369 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59075c5-1ae4-4e59-8785-01acd7dc5e08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=243e195c7e8c4d228c3d33c874184a68)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:20.700681 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59075c5-1ae4-4e59-8785-01acd7dc5e08 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:20.758386 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b59075c5-1ae4-4e59-8785-01acd7dc5e08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '243e195c7e8c4d228c3d33c874184a68', 'request_id': 'req-b59075c5-1ae4-4e59-8785-01acd7dc5e08'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:20.766779 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b59075c5-1ae4-4e59-8785-01acd7dc5e08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '243e195c7e8c4d228c3d33c874184a68', 'request_id': 'req-b59075c5-1ae4-4e59-8785-01acd7dc5e08'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:20.774996 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b59075c5-1ae4-4e59-8785-01acd7dc5e08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '243e195c7e8c4d228c3d33c874184a68', 'request_id': 'req-b59075c5-1ae4-4e59-8785-01acd7dc5e08'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:20.793436 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b59075c5-1ae4-4e59-8785-01acd7dc5e08 admin admin] The token cache is being invalidate because project 243e195c7e8c4d228c3d33c874184a68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:00:20.793750 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b59075c5-1ae4-4e59-8785-01acd7dc5e08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:20.794451 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b59075c5-1ae4-4e59-8785-01acd7dc5e08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:20.796001 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1094/2172] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:00:20 2022] DELETE /identity/v3/projects/243e195c7e8c4d228c3d33c874184a68 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:00:20.800186 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1db0678-4f3a-4a71-8f31-25abbacdca2a service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:20.810998 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1db0678-4f3a-4a71-8f31-25abbacdca2a service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:20.874057 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6704da-eca4-43fd-8a08-38870471d10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:20.875016 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6704da-eca4-43fd-8a08-38870471d10d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:20.875324 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6704da-eca4-43fd-8a08-38870471d10d admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:20.875569 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6704da-eca4-43fd-8a08-38870471d10d admin admin] PATH_INFO: `/v3/users/f4a7f5c4263e4faab215739da9593687` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:20.876803 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6704da-eca4-43fd-8a08-38870471d10d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4a7f5c4263e4faab215739da9593687)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:20.878746 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6704da-eca4-43fd-8a08-38870471d10d admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:20.937901 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-4b6704da-eca4-43fd-8a08-38870471d10d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4a7f5c4263e4faab215739da9593687'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:20.944743 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-4b6704da-eca4-43fd-8a08-38870471d10d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4a7f5c4263e4faab215739da9593687'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:20.957418 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-4b6704da-eca4-43fd-8a08-38870471d10d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4a7f5c4263e4faab215739da9593687'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:20.965639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-4b6704da-eca4-43fd-8a08-38870471d10d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4a7f5c4263e4faab215739da9593687'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:20.966351 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-4b6704da-eca4-43fd-8a08-38870471d10d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4a7f5c4263e4faab215739da9593687'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:20.986887 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1079/2173] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:00:20 2022] DELETE /identity/v3/users/f4a7f5c4263e4faab215739da9593687 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:00:21.400060 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59075c5-1ae4-4e59-8785-01acd7dc5e08 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:21.403091 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-b59075c5-1ae4-4e59-8785-01acd7dc5e08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:21.449906 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59075c5-1ae4-4e59-8785-01acd7dc5e08 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:21.520844 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70fda50-d3be-4b1d-93a4-103923362904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:21.521665 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70fda50-d3be-4b1d-93a4-103923362904 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:21.521953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70fda50-d3be-4b1d-93a4-103923362904 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:21.522230 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70fda50-d3be-4b1d-93a4-103923362904 admin admin] PATH_INFO: `/v3/projects/f0c3a529ee86480b80fcc17ded77a46f` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:21.523455 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70fda50-d3be-4b1d-93a4-103923362904 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0c3a529ee86480b80fcc17ded77a46f)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:21.525669 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70fda50-d3be-4b1d-93a4-103923362904 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:21.586344 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b70fda50-d3be-4b1d-93a4-103923362904 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0c3a529ee86480b80fcc17ded77a46f', 'request_id': 'req-b70fda50-d3be-4b1d-93a4-103923362904'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:21.595268 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b70fda50-d3be-4b1d-93a4-103923362904 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0c3a529ee86480b80fcc17ded77a46f', 'request_id': 'req-b70fda50-d3be-4b1d-93a4-103923362904'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:21.606833 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b70fda50-d3be-4b1d-93a4-103923362904 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0c3a529ee86480b80fcc17ded77a46f', 'request_id': 'req-b70fda50-d3be-4b1d-93a4-103923362904'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:21.620327 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b70fda50-d3be-4b1d-93a4-103923362904 admin admin] The token cache is being invalidate because project f0c3a529ee86480b80fcc17ded77a46f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:00:21.621016 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b70fda50-d3be-4b1d-93a4-103923362904 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:21.621920 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b70fda50-d3be-4b1d-93a4-103923362904 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:21.626492 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1095/2174] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:00:21 2022] DELETE /identity/v3/projects/f0c3a529ee86480b80fcc17ded77a46f => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:00:21.627869 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6704da-eca4-43fd-8a08-38870471d10d admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:21.629854 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-4b6704da-eca4-43fd-8a08-38870471d10d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:21.645276 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6704da-eca4-43fd-8a08-38870471d10d admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:21.731090 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f2a29e-7a5d-470c-9f79-c61766c3a38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:21.732000 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2a29e-7a5d-470c-9f79-c61766c3a38c admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:21.732278 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2a29e-7a5d-470c-9f79-c61766c3a38c admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:21.732583 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2a29e-7a5d-470c-9f79-c61766c3a38c admin admin] PATH_INFO: `/v3/users/257a348dd3b94e98972202bdabfbced0` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:21.733975 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f2a29e-7a5d-470c-9f79-c61766c3a38c admin admin] RBAC: Authorizing `identity:delete_user(user_id=257a348dd3b94e98972202bdabfbced0)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:21.736042 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f2a29e-7a5d-470c-9f79-c61766c3a38c admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:21.799425 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-16f2a29e-7a5d-470c-9f79-c61766c3a38c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '257a348dd3b94e98972202bdabfbced0'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:21.800145 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-16f2a29e-7a5d-470c-9f79-c61766c3a38c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '257a348dd3b94e98972202bdabfbced0'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:21.809861 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-16f2a29e-7a5d-470c-9f79-c61766c3a38c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '257a348dd3b94e98972202bdabfbced0'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:21.817422 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-16f2a29e-7a5d-470c-9f79-c61766c3a38c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '257a348dd3b94e98972202bdabfbced0'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:21.818095 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-16f2a29e-7a5d-470c-9f79-c61766c3a38c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '257a348dd3b94e98972202bdabfbced0'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:21.835877 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1080/2175] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:00:21 2022] DELETE /identity/v3/users/257a348dd3b94e98972202bdabfbced0 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:00:22.136818 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70fda50-d3be-4b1d-93a4-103923362904 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:22.139187 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-b70fda50-d3be-4b1d-93a4-103923362904 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:22.171263 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70fda50-d3be-4b1d-93a4-103923362904 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:22.235166 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c4aea-27cf-4188-8ba5-a9544a7fd665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:22.235945 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c4aea-27cf-4188-8ba5-a9544a7fd665 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:22.236231 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c4aea-27cf-4188-8ba5-a9544a7fd665 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:22.236510 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c4aea-27cf-4188-8ba5-a9544a7fd665 admin admin] PATH_INFO: `/v3/projects/d0e7efcbf5cb43f5abbe229fd73bbc25` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:22.237761 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c4aea-27cf-4188-8ba5-a9544a7fd665 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0e7efcbf5cb43f5abbe229fd73bbc25)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:22.239661 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c4aea-27cf-4188-8ba5-a9544a7fd665 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:22.292701 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-032c4aea-27cf-4188-8ba5-a9544a7fd665 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0e7efcbf5cb43f5abbe229fd73bbc25', 'request_id': 'req-032c4aea-27cf-4188-8ba5-a9544a7fd665'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:22.298620 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-032c4aea-27cf-4188-8ba5-a9544a7fd665 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0e7efcbf5cb43f5abbe229fd73bbc25', 'request_id': 'req-032c4aea-27cf-4188-8ba5-a9544a7fd665'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:22.306659 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-032c4aea-27cf-4188-8ba5-a9544a7fd665 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0e7efcbf5cb43f5abbe229fd73bbc25', 'request_id': 'req-032c4aea-27cf-4188-8ba5-a9544a7fd665'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:22.319492 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-032c4aea-27cf-4188-8ba5-a9544a7fd665 admin admin] The token cache is being invalidate because project d0e7efcbf5cb43f5abbe229fd73bbc25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:00:22.319921 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-032c4aea-27cf-4188-8ba5-a9544a7fd665 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:22.320624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-032c4aea-27cf-4188-8ba5-a9544a7fd665 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:22.323608 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1096/2176] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:00:22 2022] DELETE /identity/v3/projects/d0e7efcbf5cb43f5abbe229fd73bbc25 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:00:22.328733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb952ea5-1908-44ff-ab6d-9c9f8de4cae4 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:22.329114 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb952ea5-1908-44ff-ab6d-9c9f8de4cae4 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:22.329467 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb952ea5-1908-44ff-ab6d-9c9f8de4cae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:22.350373 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-cb952ea5-1908-44ff-ab6d-9c9f8de4cae4 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:00:22.351597 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-cb952ea5-1908-44ff-ab6d-9c9f8de4cae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:22.433023 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1081/2177] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:00:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:00:22.437719 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556e5e0-4699-4a9d-941d-cf3a2433e19a None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:22.438010 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556e5e0-4699-4a9d-941d-cf3a2433e19a None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:22.438291 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556e5e0-4699-4a9d-941d-cf3a2433e19a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:22.463170 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-d556e5e0-4699-4a9d-941d-cf3a2433e19a None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:00:22.465070 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-d556e5e0-4699-4a9d-941d-cf3a2433e19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:22.473476 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1097/2178] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:00:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:00:22.477714 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb952ea5-1908-44ff-ab6d-9c9f8de4cae4 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:22.489189 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb952ea5-1908-44ff-ab6d-9c9f8de4cae4 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:22.494386 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e051d9-26ce-40f6-aa9c-ad26dec3bb6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:22.495696 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e051d9-26ce-40f6-aa9c-ad26dec3bb6e admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:22.495976 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e051d9-26ce-40f6-aa9c-ad26dec3bb6e admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:22.496246 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e051d9-26ce-40f6-aa9c-ad26dec3bb6e admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:22.497081 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e051d9-26ce-40f6-aa9c-ad26dec3bb6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 15:00:22.497392 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e051d9-26ce-40f6-aa9c-ad26dec3bb6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:22.498997 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e051d9-26ce-40f6-aa9c-ad26dec3bb6e admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:22.511118 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1082/2179] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 15:00:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 15:00:22.527499 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d556e5e0-4699-4a9d-941d-cf3a2433e19a None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:22.537030 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d556e5e0-4699-4a9d-941d-cf3a2433e19a None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:22.542440 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce5b993-8462-4cc7-a4c5-6e2424bd2845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:22.543258 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce5b993-8462-4cc7-a4c5-6e2424bd2845 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:22.543541 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce5b993-8462-4cc7-a4c5-6e2424bd2845 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:22.543822 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce5b993-8462-4cc7-a4c5-6e2424bd2845 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:22.544519 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce5b993-8462-4cc7-a4c5-6e2424bd2845 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:22.546523 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce5b993-8462-4cc7-a4c5-6e2424bd2845 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:22.573531 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1098/2180] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:00:22 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:00:22.577268 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e051d9-26ce-40f6-aa9c-ad26dec3bb6e admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:22.589180 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e051d9-26ce-40f6-aa9c-ad26dec3bb6e admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:22.652091 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a6057a-8177-46a1-a524-4de6040ec3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:22.652802 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6057a-8177-46a1-a524-4de6040ec3ea admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:22.653091 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6057a-8177-46a1-a524-4de6040ec3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:22.653369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6057a-8177-46a1-a524-4de6040ec3ea admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:22.654073 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a6057a-8177-46a1-a524-4de6040ec3ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:22.656276 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a6057a-8177-46a1-a524-4de6040ec3ea admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:22.686463 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1083/2181] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:00:22 2022] POST /identity/v3/users => generated 286 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:00:22.690623 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce5b993-8462-4cc7-a4c5-6e2424bd2845 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:22.707769 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce5b993-8462-4cc7-a4c5-6e2424bd2845 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:22.718743 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e82ab8-166c-404d-8ab4-46d870549151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:22.719700 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e82ab8-166c-404d-8ab4-46d870549151 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:22.720213 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e82ab8-166c-404d-8ab4-46d870549151 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:22.720627 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e82ab8-166c-404d-8ab4-46d870549151 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:22.721480 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e82ab8-166c-404d-8ab4-46d870549151 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:22.723330 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e82ab8-166c-404d-8ab4-46d870549151 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:22.739634 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1099/2182] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:00:22 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:00:22.742341 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a6057a-8177-46a1-a524-4de6040ec3ea admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:22.754115 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a6057a-8177-46a1-a524-4de6040ec3ea admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:22.759887 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f528b-0c36-4ac8-9a75-d17bbb4845b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:22.760651 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f528b-0c36-4ac8-9a75-d17bbb4845b3 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:22.760975 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f528b-0c36-4ac8-9a75-d17bbb4845b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:22.761340 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f528b-0c36-4ac8-9a75-d17bbb4845b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:22.762103 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65f528b-0c36-4ac8-9a75-d17bbb4845b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:22.763723 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65f528b-0c36-4ac8-9a75-d17bbb4845b3 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:22.781713 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1084/2183] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:00:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:00:22.783239 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e82ab8-166c-404d-8ab4-46d870549151 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:22.801786 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e82ab8-166c-404d-8ab4-46d870549151 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:22.811188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de09f2d7-95a1-46f5-8973-a2a5aa07eb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:22.812754 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de09f2d7-95a1-46f5-8973-a2a5aa07eb9e admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:22.813188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de09f2d7-95a1-46f5-8973-a2a5aa07eb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:22.813639 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de09f2d7-95a1-46f5-8973-a2a5aa07eb9e admin admin] PATH_INFO: `/v3/projects/8199821451d1492aaf260e2bc556f580/users/ce54cb83c2214551b0d472a960194ffb/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:22.832131 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de09f2d7-95a1-46f5-8973-a2a5aa07eb9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8199821451d1492aaf260e2bc556f580, user_id=ce54cb83c2214551b0d472a960194ffb, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:22.835246 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de09f2d7-95a1-46f5-8973-a2a5aa07eb9e admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:22.856268 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1100/2184] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:00:22 2022] PUT /identity/v3/projects/8199821451d1492aaf260e2bc556f580/users/ce54cb83c2214551b0d472a960194ffb/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:00:22.867492 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c275b6-2144-4850-8a79-f9819c687baf None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:22.867991 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c275b6-2144-4850-8a79-f9819c687baf None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:22.868234 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c275b6-2144-4850-8a79-f9819c687baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:22.904595 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-88c275b6-2144-4850-8a79-f9819c687baf None None] MFA Rules not processed for user `ce54cb83c2214551b0d472a960194ffb`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:00:22.906138 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-88c275b6-2144-4850-8a79-f9819c687baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:22.985594 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1085/2185] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:00:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:00:23.406073 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de09f2d7-95a1-46f5-8973-a2a5aa07eb9e admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:23.408327 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-de09f2d7-95a1-46f5-8973-a2a5aa07eb9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:23.492513 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de09f2d7-95a1-46f5-8973-a2a5aa07eb9e admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:23.498521 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e9a0d-2a29-494b-8088-066daab7baf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:23.499560 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e9a0d-2a29-494b-8088-066daab7baf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:23.500400 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e9a0d-2a29-494b-8088-066daab7baf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:23.501027 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e9a0d-2a29-494b-8088-066daab7baf6 admin admin] PATH_INFO: `/v3/users/0ce5ddf308894e4faf16b8d186151371` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:23.504363 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142e9a0d-2a29-494b-8088-066daab7baf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ce5ddf308894e4faf16b8d186151371)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:23.505008 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142e9a0d-2a29-494b-8088-066daab7baf6 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:23.571945 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-142e9a0d-2a29-494b-8088-066daab7baf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ce5ddf308894e4faf16b8d186151371'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:23.582171 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-142e9a0d-2a29-494b-8088-066daab7baf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ce5ddf308894e4faf16b8d186151371'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:23.590690 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-142e9a0d-2a29-494b-8088-066daab7baf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ce5ddf308894e4faf16b8d186151371'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:23.591602 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-142e9a0d-2a29-494b-8088-066daab7baf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ce5ddf308894e4faf16b8d186151371'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:23.592210 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-142e9a0d-2a29-494b-8088-066daab7baf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ce5ddf308894e4faf16b8d186151371'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:23.613105 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1101/2186] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:00:23 2022] DELETE /identity/v3/users/0ce5ddf308894e4faf16b8d186151371 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:00:23.617693 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237b345-0da9-4808-bcce-7edd59eee413 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:23.617985 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237b345-0da9-4808-bcce-7edd59eee413 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:23.618292 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237b345-0da9-4808-bcce-7edd59eee413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:23.648325 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-f237b345-0da9-4808-bcce-7edd59eee413 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:00:23.650196 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-f237b345-0da9-4808-bcce-7edd59eee413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:23.737468 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1086/2187] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:00:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 15:00:23.752313 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e9a0d-2a29-494b-8088-066daab7baf6 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:23.753816 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-142e9a0d-2a29-494b-8088-066daab7baf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:23.829332 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e9a0d-2a29-494b-8088-066daab7baf6 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:23.833875 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d7716-e54a-4b8b-b168-52a9ecd7b7a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:23.834688 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d7716-e54a-4b8b-b168-52a9ecd7b7a4 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:23.835151 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d7716-e54a-4b8b-b168-52a9ecd7b7a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:23.835434 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d7716-e54a-4b8b-b168-52a9ecd7b7a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:23.847720 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838d7716-e54a-4b8b-b168-52a9ecd7b7a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:23.849252 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838d7716-e54a-4b8b-b168-52a9ecd7b7a4 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:23.860937 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1102/2188] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:00:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:00:24.308908 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f237b345-0da9-4808-bcce-7edd59eee413 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:24.310448 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-f237b345-0da9-4808-bcce-7edd59eee413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:24.326195 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f237b345-0da9-4808-bcce-7edd59eee413 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:24.328771 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3615f43b-93bc-44f8-b5a1-3ac63a4f1b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:24.329643 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3615f43b-93bc-44f8-b5a1-3ac63a4f1b4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:24.336077 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3615f43b-93bc-44f8-b5a1-3ac63a4f1b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:24.336479 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3615f43b-93bc-44f8-b5a1-3ac63a4f1b4f admin admin] PATH_INFO: `/v3/projects/e6ccaaa7c7734c6299a0afe65d1fdabb` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:24.338048 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3615f43b-93bc-44f8-b5a1-3ac63a4f1b4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6ccaaa7c7734c6299a0afe65d1fdabb)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:24.340704 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3615f43b-93bc-44f8-b5a1-3ac63a4f1b4f admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:24.406712 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-3615f43b-93bc-44f8-b5a1-3ac63a4f1b4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6ccaaa7c7734c6299a0afe65d1fdabb', 'request_id': 'req-3615f43b-93bc-44f8-b5a1-3ac63a4f1b4f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:24.412701 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-3615f43b-93bc-44f8-b5a1-3ac63a4f1b4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6ccaaa7c7734c6299a0afe65d1fdabb', 'request_id': 'req-3615f43b-93bc-44f8-b5a1-3ac63a4f1b4f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:24.420993 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-3615f43b-93bc-44f8-b5a1-3ac63a4f1b4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6ccaaa7c7734c6299a0afe65d1fdabb', 'request_id': 'req-3615f43b-93bc-44f8-b5a1-3ac63a4f1b4f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:24.435543 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-3615f43b-93bc-44f8-b5a1-3ac63a4f1b4f admin admin] The token cache is being invalidate because project e6ccaaa7c7734c6299a0afe65d1fdabb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:00:24.435830 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-3615f43b-93bc-44f8-b5a1-3ac63a4f1b4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:24.436480 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-3615f43b-93bc-44f8-b5a1-3ac63a4f1b4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:24.438048 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1087/2189] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:00:24 2022] DELETE /identity/v3/projects/e6ccaaa7c7734c6299a0afe65d1fdabb => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:00:24.442196 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d7716-e54a-4b8b-b168-52a9ecd7b7a4 service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:24.453306 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d7716-e54a-4b8b-b168-52a9ecd7b7a4 service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:24.515725 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c191ebf0-c2e6-468d-9325-d9da4b57d957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:24.516126 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c191ebf0-c2e6-468d-9325-d9da4b57d957 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:24.516406 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c191ebf0-c2e6-468d-9325-d9da4b57d957 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:24.517764 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c191ebf0-c2e6-468d-9325-d9da4b57d957 admin admin] PATH_INFO: `/v3/users/3dcef02cb0b04fc18f821bbec9f3a307` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:24.518625 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c191ebf0-c2e6-468d-9325-d9da4b57d957 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dcef02cb0b04fc18f821bbec9f3a307)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:24.520542 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c191ebf0-c2e6-468d-9325-d9da4b57d957 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:24.568648 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c191ebf0-c2e6-468d-9325-d9da4b57d957 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dcef02cb0b04fc18f821bbec9f3a307'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:24.576116 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c191ebf0-c2e6-468d-9325-d9da4b57d957 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dcef02cb0b04fc18f821bbec9f3a307'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:24.583601 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c191ebf0-c2e6-468d-9325-d9da4b57d957 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dcef02cb0b04fc18f821bbec9f3a307'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:24.584256 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c191ebf0-c2e6-468d-9325-d9da4b57d957 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dcef02cb0b04fc18f821bbec9f3a307'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:24.585182 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c191ebf0-c2e6-468d-9325-d9da4b57d957 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dcef02cb0b04fc18f821bbec9f3a307'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:24.604069 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1103/2190] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:00:24 2022] DELETE /identity/v3/users/3dcef02cb0b04fc18f821bbec9f3a307 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:00:24.952252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3615f43b-93bc-44f8-b5a1-3ac63a4f1b4f admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:24.954220 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-3615f43b-93bc-44f8-b5a1-3ac63a4f1b4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:24.974321 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3615f43b-93bc-44f8-b5a1-3ac63a4f1b4f admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:25.038620 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de69bb8-025a-472b-9509-c74bc742a9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:25.039686 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de69bb8-025a-472b-9509-c74bc742a9bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:25.040060 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de69bb8-025a-472b-9509-c74bc742a9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:25.040473 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de69bb8-025a-472b-9509-c74bc742a9bf admin admin] PATH_INFO: `/v3/projects/07fe248917ae4b468a011e204d9b9003` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:25.042400 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de69bb8-025a-472b-9509-c74bc742a9bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=07fe248917ae4b468a011e204d9b9003)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:25.044594 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de69bb8-025a-472b-9509-c74bc742a9bf admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:25.092756 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8de69bb8-025a-472b-9509-c74bc742a9bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07fe248917ae4b468a011e204d9b9003', 'request_id': 'req-8de69bb8-025a-472b-9509-c74bc742a9bf'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:25.099467 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8de69bb8-025a-472b-9509-c74bc742a9bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07fe248917ae4b468a011e204d9b9003', 'request_id': 'req-8de69bb8-025a-472b-9509-c74bc742a9bf'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:25.108347 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8de69bb8-025a-472b-9509-c74bc742a9bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07fe248917ae4b468a011e204d9b9003', 'request_id': 'req-8de69bb8-025a-472b-9509-c74bc742a9bf'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:25.120327 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8de69bb8-025a-472b-9509-c74bc742a9bf admin admin] The token cache is being invalidate because project 07fe248917ae4b468a011e204d9b9003 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:00:25.120729 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8de69bb8-025a-472b-9509-c74bc742a9bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:25.121271 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8de69bb8-025a-472b-9509-c74bc742a9bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:25.122835 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1088/2191] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:00:24 2022] DELETE /identity/v3/projects/07fe248917ae4b468a011e204d9b9003 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:00:25.128029 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df59344-0c11-43ac-b703-1579f9a657ba None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:25.128507 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df59344-0c11-43ac-b703-1579f9a657ba None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:25.128884 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df59344-0c11-43ac-b703-1579f9a657ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:25.156998 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-2df59344-0c11-43ac-b703-1579f9a657ba None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:00:25.156998 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-2df59344-0c11-43ac-b703-1579f9a657ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:25.222961 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1104/2192] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:00:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:00:25.231629 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0a375-d14c-4413-8c01-7765fe22ac03 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:25.231948 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0a375-d14c-4413-8c01-7765fe22ac03 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:25.232190 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0a375-d14c-4413-8c01-7765fe22ac03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:25.254857 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-35c0a375-d14c-4413-8c01-7765fe22ac03 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:00:25.256049 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-35c0a375-d14c-4413-8c01-7765fe22ac03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:25.265274 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1089/2193] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:00:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 15:00:25.269821 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df59344-0c11-43ac-b703-1579f9a657ba None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:25.280541 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df59344-0c11-43ac-b703-1579f9a657ba None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:25.285703 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c82c8-e56e-4065-9de7-134c6fd6d367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:25.286375 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c82c8-e56e-4065-9de7-134c6fd6d367 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:25.286661 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c82c8-e56e-4065-9de7-134c6fd6d367 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:25.286949 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c82c8-e56e-4065-9de7-134c6fd6d367 admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:25.287679 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c82c8-e56e-4065-9de7-134c6fd6d367 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 15:00:25.287989 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c82c8-e56e-4065-9de7-134c6fd6d367 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:25.289630 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c82c8-e56e-4065-9de7-134c6fd6d367 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:25.302255 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1105/2194] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 15:00:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 15:00:25.306656 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c0a375-d14c-4413-8c01-7765fe22ac03 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:25.317770 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c0a375-d14c-4413-8c01-7765fe22ac03 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:25.322521 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392d8fd2-0433-4f75-9d45-426872912bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:25.323822 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d8fd2-0433-4f75-9d45-426872912bda admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:25.324202 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d8fd2-0433-4f75-9d45-426872912bda admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:25.324486 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d8fd2-0433-4f75-9d45-426872912bda admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:25.325213 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392d8fd2-0433-4f75-9d45-426872912bda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:25.327121 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392d8fd2-0433-4f75-9d45-426872912bda admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:25.353187 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1090/2195] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:00:25 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:00:25.357606 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c82c8-e56e-4065-9de7-134c6fd6d367 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:25.370894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c82c8-e56e-4065-9de7-134c6fd6d367 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:25.432933 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa9c446-fc94-49a8-85f8-5a09ee0535b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:25.433614 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9c446-fc94-49a8-85f8-5a09ee0535b3 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:25.433901 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9c446-fc94-49a8-85f8-5a09ee0535b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:25.434175 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9c446-fc94-49a8-85f8-5a09ee0535b3 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:25.434865 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa9c446-fc94-49a8-85f8-5a09ee0535b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:25.436870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa9c446-fc94-49a8-85f8-5a09ee0535b3 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:25.458167 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1106/2196] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:00:25 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:00:25.462453 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392d8fd2-0433-4f75-9d45-426872912bda admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:25.480474 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392d8fd2-0433-4f75-9d45-426872912bda admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:25.485562 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0482d27a-6ab6-4cf8-a68a-c6d5d1413933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:25.486273 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0482d27a-6ab6-4cf8-a68a-c6d5d1413933 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:25.486559 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0482d27a-6ab6-4cf8-a68a-c6d5d1413933 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:25.486847 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0482d27a-6ab6-4cf8-a68a-c6d5d1413933 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:25.487580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0482d27a-6ab6-4cf8-a68a-c6d5d1413933 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:25.489229 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0482d27a-6ab6-4cf8-a68a-c6d5d1413933 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:25.503292 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1091/2197] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:00:25.504587 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa9c446-fc94-49a8-85f8-5a09ee0535b3 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:25.514796 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa9c446-fc94-49a8-85f8-5a09ee0535b3 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:25.519607 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4405e3ee-7df1-4fd1-8547-79c2e12a7b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:25.520280 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405e3ee-7df1-4fd1-8547-79c2e12a7b14 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:25.520565 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405e3ee-7df1-4fd1-8547-79c2e12a7b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:25.520885 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405e3ee-7df1-4fd1-8547-79c2e12a7b14 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:25.521645 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4405e3ee-7df1-4fd1-8547-79c2e12a7b14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:25.523252 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4405e3ee-7df1-4fd1-8547-79c2e12a7b14 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:25.537864 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1107/2198] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:00:25.540022 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0482d27a-6ab6-4cf8-a68a-c6d5d1413933 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:25.556505 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0482d27a-6ab6-4cf8-a68a-c6d5d1413933 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:25.564674 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692171fe-9b79-4088-8ce9-301ea522144c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:25.565677 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692171fe-9b79-4088-8ce9-301ea522144c admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:25.565962 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692171fe-9b79-4088-8ce9-301ea522144c admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:25.566233 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692171fe-9b79-4088-8ce9-301ea522144c admin admin] PATH_INFO: `/v3/projects/a85fefbbc677485a804146c813e2ced1/users/57d0d97efe544b4a9cc8c644d0cf56ad/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:25.585899 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692171fe-9b79-4088-8ce9-301ea522144c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a85fefbbc677485a804146c813e2ced1, user_id=57d0d97efe544b4a9cc8c644d0cf56ad, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:25.588589 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692171fe-9b79-4088-8ce9-301ea522144c admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:25.602481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1092/2199] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:00:25 2022] PUT /identity/v3/projects/a85fefbbc677485a804146c813e2ced1/users/57d0d97efe544b4a9cc8c644d0cf56ad/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:00:25.620793 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223de559-a189-4f65-bcd3-aeb9623a7c97 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:25.621122 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223de559-a189-4f65-bcd3-aeb9623a7c97 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:25.621411 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223de559-a189-4f65-bcd3-aeb9623a7c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:25.658499 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-223de559-a189-4f65-bcd3-aeb9623a7c97 None None] MFA Rules not processed for user `57d0d97efe544b4a9cc8c644d0cf56ad`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:00:25.659760 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-223de559-a189-4f65-bcd3-aeb9623a7c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:25.725814 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1108/2200] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:00:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:00:25.731120 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692171fe-9b79-4088-8ce9-301ea522144c admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:25.803889 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692171fe-9b79-4088-8ce9-301ea522144c admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:25.809089 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b968ecd-a009-493d-9fc8-2215d6568bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:25.809758 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b968ecd-a009-493d-9fc8-2215d6568bf3 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:25.810043 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b968ecd-a009-493d-9fc8-2215d6568bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:25.810378 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b968ecd-a009-493d-9fc8-2215d6568bf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:25.811004 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b968ecd-a009-493d-9fc8-2215d6568bf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:25.812921 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b968ecd-a009-493d-9fc8-2215d6568bf3 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:25.843599 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1093/2201] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:00:25 2022] POST /identity/v3/projects => generated 387 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:00:25.851600 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223de559-a189-4f65-bcd3-aeb9623a7c97 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:25.860498 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223de559-a189-4f65-bcd3-aeb9623a7c97 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:25.923075 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac86fcb-8842-4182-a36e-d1498dda149d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:25.924155 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac86fcb-8842-4182-a36e-d1498dda149d admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:25.924755 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac86fcb-8842-4182-a36e-d1498dda149d admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:25.925206 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac86fcb-8842-4182-a36e-d1498dda149d admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:25.926368 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac86fcb-8842-4182-a36e-d1498dda149d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:25.930101 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac86fcb-8842-4182-a36e-d1498dda149d admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:25.958978 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1109/2202] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:00:25 2022] POST /identity/v3/users => generated 300 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 15:00:25.960856 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b968ecd-a009-493d-9fc8-2215d6568bf3 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:25.971451 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b968ecd-a009-493d-9fc8-2215d6568bf3 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:25.977690 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee12e03-cfcc-4823-b174-e0525850f400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:25.978577 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee12e03-cfcc-4823-b174-e0525850f400 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:25.979500 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee12e03-cfcc-4823-b174-e0525850f400 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:25.980057 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee12e03-cfcc-4823-b174-e0525850f400 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:25.980818 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee12e03-cfcc-4823-b174-e0525850f400 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:25.982581 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee12e03-cfcc-4823-b174-e0525850f400 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:25.993885 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1094/2203] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:00:26.001578 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac86fcb-8842-4182-a36e-d1498dda149d admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:26.013950 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac86fcb-8842-4182-a36e-d1498dda149d admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:26.020255 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5741c32a-b348-40db-9ede-a2960565563e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:26.021356 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741c32a-b348-40db-9ede-a2960565563e admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:26.021671 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741c32a-b348-40db-9ede-a2960565563e admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:26.021953 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741c32a-b348-40db-9ede-a2960565563e admin admin] PATH_INFO: `/v3/projects/bddc5a695669419182489de188beba66/users/e36f0a75918149b5b01da75667c78536/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:26.040210 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5741c32a-b348-40db-9ede-a2960565563e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bddc5a695669419182489de188beba66, user_id=e36f0a75918149b5b01da75667c78536, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:26.043557 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5741c32a-b348-40db-9ede-a2960565563e admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:26.067400 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1110/2204] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:00:25 2022] PUT /identity/v3/projects/bddc5a695669419182489de188beba66/users/e36f0a75918149b5b01da75667c78536/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:00:26.071669 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee12e03-cfcc-4823-b174-e0525850f400 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:26.082639 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee12e03-cfcc-4823-b174-e0525850f400 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:26.160790 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447a958e-3b68-4d5d-8ef5-2778f60d08e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:26.161470 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a958e-3b68-4d5d-8ef5-2778f60d08e1 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:26.161752 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a958e-3b68-4d5d-8ef5-2778f60d08e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:26.162032 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a958e-3b68-4d5d-8ef5-2778f60d08e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:26.162996 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447a958e-3b68-4d5d-8ef5-2778f60d08e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:26.164390 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447a958e-3b68-4d5d-8ef5-2778f60d08e1 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:26.175644 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1095/2205] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:00:26 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:00:26.185530 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5741c32a-b348-40db-9ede-a2960565563e admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:26.193623 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5741c32a-b348-40db-9ede-a2960565563e admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:26.201923 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3739bb-171b-42ac-8a09-66c13c89aed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:26.202823 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3739bb-171b-42ac-8a09-66c13c89aed2 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:26.203105 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3739bb-171b-42ac-8a09-66c13c89aed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:26.203381 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3739bb-171b-42ac-8a09-66c13c89aed2 admin admin] PATH_INFO: `/v3/projects/bddc5a695669419182489de188beba66/users/e36f0a75918149b5b01da75667c78536/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:26.206273 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3739bb-171b-42ac-8a09-66c13c89aed2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bddc5a695669419182489de188beba66, user_id=e36f0a75918149b5b01da75667c78536, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:26.211007 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3739bb-171b-42ac-8a09-66c13c89aed2 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:26.223105 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1111/2206] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:00:26 2022] PUT /identity/v3/projects/bddc5a695669419182489de188beba66/users/e36f0a75918149b5b01da75667c78536/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:00:26.236305 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa6706-63d9-42d7-8776-6b5d7c91ce07 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:26.236809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa6706-63d9-42d7-8776-6b5d7c91ce07 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:26.237144 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa6706-63d9-42d7-8776-6b5d7c91ce07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:26.267909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-f6aa6706-63d9-42d7-8776-6b5d7c91ce07 None None] MFA Rules not processed for user `e36f0a75918149b5b01da75667c78536`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:00:26.269181 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-f6aa6706-63d9-42d7-8776-6b5d7c91ce07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:26.351145 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1096/2207] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:00:26 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:00:26.362533 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3739bb-171b-42ac-8a09-66c13c89aed2 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:26.364339 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-7b3739bb-171b-42ac-8a09-66c13c89aed2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:26.425279 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3739bb-171b-42ac-8a09-66c13c89aed2 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:26.435531 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f8e8e3-b5ab-4c15-8100-2991e193dbd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:26.436205 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f8e8e3-b5ab-4c15-8100-2991e193dbd2 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:26.436490 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f8e8e3-b5ab-4c15-8100-2991e193dbd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:26.436799 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f8e8e3-b5ab-4c15-8100-2991e193dbd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:26.446651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f8e8e3-b5ab-4c15-8100-2991e193dbd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:26.448296 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f8e8e3-b5ab-4c15-8100-2991e193dbd2 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:26.458401 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1112/2208] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:00:26 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:00:27.095575 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aa6706-63d9-42d7-8776-6b5d7c91ce07 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:27.163173 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aa6706-63d9-42d7-8776-6b5d7c91ce07 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:27.168250 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acf7ee6-7d5c-4857-a23a-50c0e9b5092c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:27.169219 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf7ee6-7d5c-4857-a23a-50c0e9b5092c admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:27.169507 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf7ee6-7d5c-4857-a23a-50c0e9b5092c admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:27.169784 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf7ee6-7d5c-4857-a23a-50c0e9b5092c admin admin] PATH_INFO: `/v3/users/57d0d97efe544b4a9cc8c644d0cf56ad` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:27.171017 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acf7ee6-7d5c-4857-a23a-50c0e9b5092c admin admin] RBAC: Authorizing `identity:delete_user(user_id=57d0d97efe544b4a9cc8c644d0cf56ad)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:27.173246 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acf7ee6-7d5c-4857-a23a-50c0e9b5092c admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:27.239867 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-1acf7ee6-7d5c-4857-a23a-50c0e9b5092c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57d0d97efe544b4a9cc8c644d0cf56ad'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:27.240550 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-1acf7ee6-7d5c-4857-a23a-50c0e9b5092c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57d0d97efe544b4a9cc8c644d0cf56ad'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:27.249129 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-1acf7ee6-7d5c-4857-a23a-50c0e9b5092c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57d0d97efe544b4a9cc8c644d0cf56ad'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:27.256205 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-1acf7ee6-7d5c-4857-a23a-50c0e9b5092c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57d0d97efe544b4a9cc8c644d0cf56ad'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:27.256914 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-1acf7ee6-7d5c-4857-a23a-50c0e9b5092c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57d0d97efe544b4a9cc8c644d0cf56ad'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:27.274338 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1097/2209] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:00:27 2022] DELETE /identity/v3/users/57d0d97efe544b4a9cc8c644d0cf56ad => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:00:27.281913 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440af5b-34f2-4bdf-a39e-f7aece21c41a None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:27.282198 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440af5b-34f2-4bdf-a39e-f7aece21c41a None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:27.282478 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440af5b-34f2-4bdf-a39e-f7aece21c41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:27.301019 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-7440af5b-34f2-4bdf-a39e-f7aece21c41a None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:00:27.302212 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-7440af5b-34f2-4bdf-a39e-f7aece21c41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:27.361949 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1113/2210] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:00:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:00:27.373996 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acf7ee6-7d5c-4857-a23a-50c0e9b5092c admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:27.375484 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-1acf7ee6-7d5c-4857-a23a-50c0e9b5092c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:27.436498 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acf7ee6-7d5c-4857-a23a-50c0e9b5092c admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:27.440092 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591018e5-2032-4e6b-b7ef-816a07824668 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:27.440801 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591018e5-2032-4e6b-b7ef-816a07824668 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:27.441083 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591018e5-2032-4e6b-b7ef-816a07824668 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:27.441355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591018e5-2032-4e6b-b7ef-816a07824668 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:27.450888 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591018e5-2032-4e6b-b7ef-816a07824668 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:27.452316 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591018e5-2032-4e6b-b7ef-816a07824668 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:27.462094 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1098/2211] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:00:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:00:27.754594 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7440af5b-34f2-4bdf-a39e-f7aece21c41a None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:27.756268 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-7440af5b-34f2-4bdf-a39e-f7aece21c41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:27.772506 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7440af5b-34f2-4bdf-a39e-f7aece21c41a None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:27.781469 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d4a1a-632d-4a26-883e-726d707ef69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:27.782818 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d4a1a-632d-4a26-883e-726d707ef69e admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:27.783134 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d4a1a-632d-4a26-883e-726d707ef69e admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:27.783790 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d4a1a-632d-4a26-883e-726d707ef69e admin admin] PATH_INFO: `/v3/projects/a85fefbbc677485a804146c813e2ced1` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:27.785078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d4a1a-632d-4a26-883e-726d707ef69e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a85fefbbc677485a804146c813e2ced1)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:27.787810 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d4a1a-632d-4a26-883e-726d707ef69e admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:27.853249 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-007d4a1a-632d-4a26-883e-726d707ef69e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a85fefbbc677485a804146c813e2ced1', 'request_id': 'req-007d4a1a-632d-4a26-883e-726d707ef69e'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:27.859761 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-007d4a1a-632d-4a26-883e-726d707ef69e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a85fefbbc677485a804146c813e2ced1', 'request_id': 'req-007d4a1a-632d-4a26-883e-726d707ef69e'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:27.869112 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-007d4a1a-632d-4a26-883e-726d707ef69e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a85fefbbc677485a804146c813e2ced1', 'request_id': 'req-007d4a1a-632d-4a26-883e-726d707ef69e'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:27.882258 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-007d4a1a-632d-4a26-883e-726d707ef69e admin admin] The token cache is being invalidate because project a85fefbbc677485a804146c813e2ced1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:00:27.882553 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-007d4a1a-632d-4a26-883e-726d707ef69e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:27.883224 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-007d4a1a-632d-4a26-883e-726d707ef69e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:27.885580 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1114/2212] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:00:27 2022] DELETE /identity/v3/projects/a85fefbbc677485a804146c813e2ced1 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:00:27.892213 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591018e5-2032-4e6b-b7ef-816a07824668 service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:27.903166 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591018e5-2032-4e6b-b7ef-816a07824668 service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:27.972191 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ce4c33-8a63-4d4c-aaac-32972eb7c400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:27.973642 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce4c33-8a63-4d4c-aaac-32972eb7c400 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:27.974178 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce4c33-8a63-4d4c-aaac-32972eb7c400 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:27.974699 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce4c33-8a63-4d4c-aaac-32972eb7c400 admin admin] PATH_INFO: `/v3/users/e36f0a75918149b5b01da75667c78536` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:27.976679 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ce4c33-8a63-4d4c-aaac-32972eb7c400 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e36f0a75918149b5b01da75667c78536)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:27.979055 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ce4c33-8a63-4d4c-aaac-32972eb7c400 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:28.031942 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-88ce4c33-8a63-4d4c-aaac-32972eb7c400 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e36f0a75918149b5b01da75667c78536'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:28.031942 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-88ce4c33-8a63-4d4c-aaac-32972eb7c400 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e36f0a75918149b5b01da75667c78536'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:28.038504 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-88ce4c33-8a63-4d4c-aaac-32972eb7c400 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e36f0a75918149b5b01da75667c78536'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:28.045927 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-88ce4c33-8a63-4d4c-aaac-32972eb7c400 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e36f0a75918149b5b01da75667c78536'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:28.046685 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-88ce4c33-8a63-4d4c-aaac-32972eb7c400 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e36f0a75918149b5b01da75667c78536'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:28.063804 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1099/2213] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:00:27 2022] DELETE /identity/v3/users/e36f0a75918149b5b01da75667c78536 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:00:28.372813 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d4a1a-632d-4a26-883e-726d707ef69e admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:28.376079 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-007d4a1a-632d-4a26-883e-726d707ef69e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:28.393406 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d4a1a-632d-4a26-883e-726d707ef69e admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:28.459140 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a514f-a755-4457-bd12-d2ad581e212c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:28.460018 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a514f-a755-4457-bd12-d2ad581e212c admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:28.460299 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a514f-a755-4457-bd12-d2ad581e212c admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:28.460573 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a514f-a755-4457-bd12-d2ad581e212c admin admin] PATH_INFO: `/v3/projects/bddc5a695669419182489de188beba66` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:28.461811 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158a514f-a755-4457-bd12-d2ad581e212c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bddc5a695669419182489de188beba66)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:28.464289 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158a514f-a755-4457-bd12-d2ad581e212c admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:28.512954 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-158a514f-a755-4457-bd12-d2ad581e212c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bddc5a695669419182489de188beba66', 'request_id': 'req-158a514f-a755-4457-bd12-d2ad581e212c'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:28.518356 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-158a514f-a755-4457-bd12-d2ad581e212c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bddc5a695669419182489de188beba66', 'request_id': 'req-158a514f-a755-4457-bd12-d2ad581e212c'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:28.527435 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-158a514f-a755-4457-bd12-d2ad581e212c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bddc5a695669419182489de188beba66', 'request_id': 'req-158a514f-a755-4457-bd12-d2ad581e212c'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:28.539493 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-158a514f-a755-4457-bd12-d2ad581e212c admin admin] The token cache is being invalidate because project bddc5a695669419182489de188beba66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:00:28.539786 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-158a514f-a755-4457-bd12-d2ad581e212c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:28.540465 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-158a514f-a755-4457-bd12-d2ad581e212c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:28.542611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1115/2214] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:00:28 2022] DELETE /identity/v3/projects/bddc5a695669419182489de188beba66 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:00:28.548372 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f7e5e-20c2-4f09-9296-7b137b1bc634 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:28.548870 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f7e5e-20c2-4f09-9296-7b137b1bc634 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:28.548959 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f7e5e-20c2-4f09-9296-7b137b1bc634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:28.578806 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-822f7e5e-20c2-4f09-9296-7b137b1bc634 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:00:28.580488 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-822f7e5e-20c2-4f09-9296-7b137b1bc634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:28.651215 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1100/2215] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:00:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:00:28.659573 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66daef3c-a79b-4c63-b1d4-95b7fa07dbb4 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:28.659865 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66daef3c-a79b-4c63-b1d4-95b7fa07dbb4 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:28.660149 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66daef3c-a79b-4c63-b1d4-95b7fa07dbb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:28.678994 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-66daef3c-a79b-4c63-b1d4-95b7fa07dbb4 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:00:28.680215 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-66daef3c-a79b-4c63-b1d4-95b7fa07dbb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:28.688836 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1116/2216] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:00:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:00:28.693438 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822f7e5e-20c2-4f09-9296-7b137b1bc634 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:28.704572 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822f7e5e-20c2-4f09-9296-7b137b1bc634 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:28.708848 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e7063f-dfdf-4762-9499-f0ce190f17e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:28.709524 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e7063f-dfdf-4762-9499-f0ce190f17e1 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:28.709803 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e7063f-dfdf-4762-9499-f0ce190f17e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:28.710077 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e7063f-dfdf-4762-9499-f0ce190f17e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:28.710812 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e7063f-dfdf-4762-9499-f0ce190f17e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 15:00:28.711114 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e7063f-dfdf-4762-9499-f0ce190f17e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:28.712752 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e7063f-dfdf-4762-9499-f0ce190f17e1 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:28.725760 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1101/2217] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 15:00:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 15:00:28.730716 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66daef3c-a79b-4c63-b1d4-95b7fa07dbb4 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:28.740578 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66daef3c-a79b-4c63-b1d4-95b7fa07dbb4 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:28.746265 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40613390-26a8-4116-be84-eff65e159cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:28.746924 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40613390-26a8-4116-be84-eff65e159cba admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:28.747210 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40613390-26a8-4116-be84-eff65e159cba admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:28.747487 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40613390-26a8-4116-be84-eff65e159cba admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:28.748194 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40613390-26a8-4116-be84-eff65e159cba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:28.750188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40613390-26a8-4116-be84-eff65e159cba admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:28.777018 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1117/2218] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:00:28 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 15:00:28.781777 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e7063f-dfdf-4762-9499-f0ce190f17e1 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:28.792590 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e7063f-dfdf-4762-9499-f0ce190f17e1 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:28.863619 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb9a48-f06c-4357-8873-4fcebcb91c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:28.864284 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb9a48-f06c-4357-8873-4fcebcb91c81 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:28.864560 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb9a48-f06c-4357-8873-4fcebcb91c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:28.864861 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb9a48-f06c-4357-8873-4fcebcb91c81 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:28.865553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bb9a48-f06c-4357-8873-4fcebcb91c81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:28.867446 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bb9a48-f06c-4357-8873-4fcebcb91c81 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:28.897275 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1102/2219] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:00:28 2022] POST /identity/v3/users => generated 287 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:00:28.898628 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40613390-26a8-4116-be84-eff65e159cba admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:28.909382 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40613390-26a8-4116-be84-eff65e159cba admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:28.914858 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e469dcd-ce18-4420-803d-4cd662c1fae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:28.915532 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e469dcd-ce18-4420-803d-4cd662c1fae8 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:28.915812 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e469dcd-ce18-4420-803d-4cd662c1fae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:28.916590 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e469dcd-ce18-4420-803d-4cd662c1fae8 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:28.917695 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e469dcd-ce18-4420-803d-4cd662c1fae8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:28.919216 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e469dcd-ce18-4420-803d-4cd662c1fae8 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:28.932909 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1118/2220] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:00:28.936733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb9a48-f06c-4357-8873-4fcebcb91c81 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:28.946614 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb9a48-f06c-4357-8873-4fcebcb91c81 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:28.952237 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f986a6-ee23-4975-9222-d5cdb6b91feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:28.952953 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f986a6-ee23-4975-9222-d5cdb6b91feb admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:28.953234 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f986a6-ee23-4975-9222-d5cdb6b91feb admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:28.953512 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f986a6-ee23-4975-9222-d5cdb6b91feb admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:28.954262 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f986a6-ee23-4975-9222-d5cdb6b91feb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:28.955857 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f986a6-ee23-4975-9222-d5cdb6b91feb admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:28.967223 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1103/2221] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:00:28.971427 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e469dcd-ce18-4420-803d-4cd662c1fae8 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:28.982361 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e469dcd-ce18-4420-803d-4cd662c1fae8 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:28.987862 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eaf8ed-484d-4a84-b6db-d9d5d002e3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:28.988786 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eaf8ed-484d-4a84-b6db-d9d5d002e3a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:28.989068 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eaf8ed-484d-4a84-b6db-d9d5d002e3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:28.989438 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eaf8ed-484d-4a84-b6db-d9d5d002e3a5 admin admin] PATH_INFO: `/v3/projects/dc8f2b033d52449194bf8a4f47752cfd/users/4156899c76974e59934ae0542e1c9807/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:29.008113 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eaf8ed-484d-4a84-b6db-d9d5d002e3a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc8f2b033d52449194bf8a4f47752cfd, user_id=4156899c76974e59934ae0542e1c9807, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:29.011584 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eaf8ed-484d-4a84-b6db-d9d5d002e3a5 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:29.047044 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1119/2222] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:00:28 2022] PUT /identity/v3/projects/dc8f2b033d52449194bf8a4f47752cfd/users/4156899c76974e59934ae0542e1c9807/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:00:29.049895 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c157ff-f194-4254-8f15-090a2c779256 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:29.050406 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c157ff-f194-4254-8f15-090a2c779256 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:29.050834 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c157ff-f194-4254-8f15-090a2c779256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:29.090655 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-91c157ff-f194-4254-8f15-090a2c779256 None None] MFA Rules not processed for user `4156899c76974e59934ae0542e1c9807`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:00:29.091887 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-91c157ff-f194-4254-8f15-090a2c779256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:29.188155 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1104/2223] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:00:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:00:29.199961 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eaf8ed-484d-4a84-b6db-d9d5d002e3a5 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:29.201832 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-77eaf8ed-484d-4a84-b6db-d9d5d002e3a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:29.264297 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eaf8ed-484d-4a84-b6db-d9d5d002e3a5 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:29.268033 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb1ac69-f9d2-4d9e-ab4a-246ffeff4c4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:29.268704 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb1ac69-f9d2-4d9e-ab4a-246ffeff4c4a service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:29.268985 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb1ac69-f9d2-4d9e-ab4a-246ffeff4c4a service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:29.269255 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb1ac69-f9d2-4d9e-ab4a-246ffeff4c4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:29.279145 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb1ac69-f9d2-4d9e-ab4a-246ffeff4c4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:29.281967 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb1ac69-f9d2-4d9e-ab4a-246ffeff4c4a service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:29.292896 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1120/2224] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:00:29 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:00:29.313960 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c157ff-f194-4254-8f15-090a2c779256 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:29.388798 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c157ff-f194-4254-8f15-090a2c779256 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:29.393755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09be8e6-06e6-4ed8-b7fe-f57c63ba03a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:29.394815 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09be8e6-06e6-4ed8-b7fe-f57c63ba03a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:29.395094 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09be8e6-06e6-4ed8-b7fe-f57c63ba03a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:29.395360 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09be8e6-06e6-4ed8-b7fe-f57c63ba03a9 admin admin] PATH_INFO: `/v3/users/4156899c76974e59934ae0542e1c9807` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:29.396635 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09be8e6-06e6-4ed8-b7fe-f57c63ba03a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4156899c76974e59934ae0542e1c9807)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:29.398567 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09be8e6-06e6-4ed8-b7fe-f57c63ba03a9 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:29.447920 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-f09be8e6-06e6-4ed8-b7fe-f57c63ba03a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4156899c76974e59934ae0542e1c9807'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:29.448663 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-f09be8e6-06e6-4ed8-b7fe-f57c63ba03a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4156899c76974e59934ae0542e1c9807'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:29.458103 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-f09be8e6-06e6-4ed8-b7fe-f57c63ba03a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4156899c76974e59934ae0542e1c9807'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:29.466288 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-f09be8e6-06e6-4ed8-b7fe-f57c63ba03a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4156899c76974e59934ae0542e1c9807'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:29.467166 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-f09be8e6-06e6-4ed8-b7fe-f57c63ba03a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4156899c76974e59934ae0542e1c9807'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:29.485656 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1105/2225] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:00:29 2022] DELETE /identity/v3/users/4156899c76974e59934ae0542e1c9807 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:00:29.490198 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49500db7-72a8-4f35-8de4-01eaf9c1e74e None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:29.490486 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49500db7-72a8-4f35-8de4-01eaf9c1e74e None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:29.490773 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49500db7-72a8-4f35-8de4-01eaf9c1e74e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:29.509221 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-49500db7-72a8-4f35-8de4-01eaf9c1e74e None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:00:29.510507 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-49500db7-72a8-4f35-8de4-01eaf9c1e74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:29.574155 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1121/2226] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:00:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:00:29.583862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09be8e6-06e6-4ed8-b7fe-f57c63ba03a9 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:29.585386 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-f09be8e6-06e6-4ed8-b7fe-f57c63ba03a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:29.646881 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09be8e6-06e6-4ed8-b7fe-f57c63ba03a9 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:29.650523 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef72526f-fc0c-433b-b0ed-3951d62994c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:29.651421 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef72526f-fc0c-433b-b0ed-3951d62994c6 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:29.651421 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef72526f-fc0c-433b-b0ed-3951d62994c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:29.651753 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef72526f-fc0c-433b-b0ed-3951d62994c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:29.662028 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef72526f-fc0c-433b-b0ed-3951d62994c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:29.663555 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef72526f-fc0c-433b-b0ed-3951d62994c6 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:29.677329 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1106/2227] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:00:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:00:29.971678 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49500db7-72a8-4f35-8de4-01eaf9c1e74e None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:29.973197 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-49500db7-72a8-4f35-8de4-01eaf9c1e74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:29.986525 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49500db7-72a8-4f35-8de4-01eaf9c1e74e None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:29.991941 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d0396-66e8-4262-95c9-ed094806a1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:29.992976 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d0396-66e8-4262-95c9-ed094806a1f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:29.993262 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d0396-66e8-4262-95c9-ed094806a1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:29.993539 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d0396-66e8-4262-95c9-ed094806a1f8 admin admin] PATH_INFO: `/v3/projects/dc8f2b033d52449194bf8a4f47752cfd` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:29.994766 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965d0396-66e8-4262-95c9-ed094806a1f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc8f2b033d52449194bf8a4f47752cfd)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:29.997656 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965d0396-66e8-4262-95c9-ed094806a1f8 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:30.050168 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-965d0396-66e8-4262-95c9-ed094806a1f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc8f2b033d52449194bf8a4f47752cfd', 'request_id': 'req-965d0396-66e8-4262-95c9-ed094806a1f8'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:30.056732 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-965d0396-66e8-4262-95c9-ed094806a1f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc8f2b033d52449194bf8a4f47752cfd', 'request_id': 'req-965d0396-66e8-4262-95c9-ed094806a1f8'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:30.066372 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-965d0396-66e8-4262-95c9-ed094806a1f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc8f2b033d52449194bf8a4f47752cfd', 'request_id': 'req-965d0396-66e8-4262-95c9-ed094806a1f8'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:30.080073 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-965d0396-66e8-4262-95c9-ed094806a1f8 admin admin] The token cache is being invalidate because project dc8f2b033d52449194bf8a4f47752cfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:00:30.080368 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-965d0396-66e8-4262-95c9-ed094806a1f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:30.081239 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-965d0396-66e8-4262-95c9-ed094806a1f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:30.082968 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1122/2228] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:00:29 2022] DELETE /identity/v3/projects/dc8f2b033d52449194bf8a4f47752cfd => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:00:30.087853 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf3bcc3-abb3-40c4-bf45-88ca05b4d3bc None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:30.088143 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf3bcc3-abb3-40c4-bf45-88ca05b4d3bc None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:30.088421 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf3bcc3-abb3-40c4-bf45-88ca05b4d3bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:30.107385 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-1bf3bcc3-abb3-40c4-bf45-88ca05b4d3bc None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:00:30.108652 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-1bf3bcc3-abb3-40c4-bf45-88ca05b4d3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:30.185788 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1107/2229] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:00:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:00:30.195691 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c6bec-8081-47b4-bcfe-f1bd8de974a5 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:30.195691 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c6bec-8081-47b4-bcfe-f1bd8de974a5 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:30.195691 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c6bec-8081-47b4-bcfe-f1bd8de974a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:30.214890 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-b63c6bec-8081-47b4-bcfe-f1bd8de974a5 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:00:30.216124 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-b63c6bec-8081-47b4-bcfe-f1bd8de974a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:30.225666 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1123/2230] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:00:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:00:30.229932 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf3bcc3-abb3-40c4-bf45-88ca05b4d3bc None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:30.240252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf3bcc3-abb3-40c4-bf45-88ca05b4d3bc None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:30.245075 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf852fc-3682-475f-aed3-b3994ad7127d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:30.245742 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf852fc-3682-475f-aed3-b3994ad7127d admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:30.249247 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf852fc-3682-475f-aed3-b3994ad7127d admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:30.249247 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf852fc-3682-475f-aed3-b3994ad7127d admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:30.249247 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf852fc-3682-475f-aed3-b3994ad7127d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 15:00:30.249247 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf852fc-3682-475f-aed3-b3994ad7127d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:30.249671 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf852fc-3682-475f-aed3-b3994ad7127d admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:30.263580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1108/2231] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 15:00:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 15:00:30.268044 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c6bec-8081-47b4-bcfe-f1bd8de974a5 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:30.278589 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c6bec-8081-47b4-bcfe-f1bd8de974a5 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:30.286912 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f332cfff-a70a-4b9a-ba6e-450cc579af91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:30.287592 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f332cfff-a70a-4b9a-ba6e-450cc579af91 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:30.287873 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f332cfff-a70a-4b9a-ba6e-450cc579af91 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:30.288147 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f332cfff-a70a-4b9a-ba6e-450cc579af91 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:30.288886 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f332cfff-a70a-4b9a-ba6e-450cc579af91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:30.290824 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f332cfff-a70a-4b9a-ba6e-450cc579af91 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:30.320642 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1124/2232] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:00:30 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 15:00:30.325655 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf852fc-3682-475f-aed3-b3994ad7127d admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:30.337731 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf852fc-3682-475f-aed3-b3994ad7127d admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:30.394513 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b14fe7a-de6a-4557-9638-c5facfb2eec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:30.395182 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14fe7a-de6a-4557-9638-c5facfb2eec2 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:30.395466 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14fe7a-de6a-4557-9638-c5facfb2eec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:30.395745 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14fe7a-de6a-4557-9638-c5facfb2eec2 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:30.396437 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b14fe7a-de6a-4557-9638-c5facfb2eec2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:30.398419 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b14fe7a-de6a-4557-9638-c5facfb2eec2 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:30.425510 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1109/2233] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:00:30 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:00:30.427377 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f332cfff-a70a-4b9a-ba6e-450cc579af91 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:30.443480 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f332cfff-a70a-4b9a-ba6e-450cc579af91 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:30.450846 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cce5f9d-8bdb-46e1-8455-dabbe43fcee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:30.450846 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce5f9d-8bdb-46e1-8455-dabbe43fcee6 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:30.450846 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce5f9d-8bdb-46e1-8455-dabbe43fcee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:30.450846 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce5f9d-8bdb-46e1-8455-dabbe43fcee6 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:30.451356 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cce5f9d-8bdb-46e1-8455-dabbe43fcee6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:30.456707 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cce5f9d-8bdb-46e1-8455-dabbe43fcee6 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:30.463574 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1125/2234] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:00:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 06 15:00:30.468377 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b14fe7a-de6a-4557-9638-c5facfb2eec2 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:30.478690 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b14fe7a-de6a-4557-9638-c5facfb2eec2 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:30.484268 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc449d7-3875-4cdd-9228-a26b82671dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:30.484968 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc449d7-3875-4cdd-9228-a26b82671dfb admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:30.485254 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc449d7-3875-4cdd-9228-a26b82671dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:30.485530 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc449d7-3875-4cdd-9228-a26b82671dfb admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:30.486251 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc449d7-3875-4cdd-9228-a26b82671dfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:30.487854 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc449d7-3875-4cdd-9228-a26b82671dfb admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:30.498340 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1110/2235] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:00:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:00:30.502671 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cce5f9d-8bdb-46e1-8455-dabbe43fcee6 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:30.514210 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cce5f9d-8bdb-46e1-8455-dabbe43fcee6 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:30.519710 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346b2f5-4b35-4bf0-a4f0-7b8d046a2fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:30.520404 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346b2f5-4b35-4bf0-a4f0-7b8d046a2fd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:30.520722 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346b2f5-4b35-4bf0-a4f0-7b8d046a2fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:30.520994 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346b2f5-4b35-4bf0-a4f0-7b8d046a2fd5 admin admin] PATH_INFO: `/v3/projects/fde1b936bf84416aa4c1f654d295de0f/users/c4152e495e9a4d9d8b25e74769781afb/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:30.537461 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8346b2f5-4b35-4bf0-a4f0-7b8d046a2fd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fde1b936bf84416aa4c1f654d295de0f, user_id=c4152e495e9a4d9d8b25e74769781afb, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:30.540435 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8346b2f5-4b35-4bf0-a4f0-7b8d046a2fd5 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:30.554855 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1126/2236] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:00:30 2022] PUT /identity/v3/projects/fde1b936bf84416aa4c1f654d295de0f/users/c4152e495e9a4d9d8b25e74769781afb/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:00:30.565952 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232bbf8c-5567-432e-b3f2-81eb5ebb18f6 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:30.566237 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232bbf8c-5567-432e-b3f2-81eb5ebb18f6 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:30.566510 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232bbf8c-5567-432e-b3f2-81eb5ebb18f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:30.601194 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-232bbf8c-5567-432e-b3f2-81eb5ebb18f6 None None] MFA Rules not processed for user `c4152e495e9a4d9d8b25e74769781afb`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:00:30.602451 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-232bbf8c-5567-432e-b3f2-81eb5ebb18f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:30.672570 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1111/2237] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:00:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:00:30.677123 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346b2f5-4b35-4bf0-a4f0-7b8d046a2fd5 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:30.740155 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346b2f5-4b35-4bf0-a4f0-7b8d046a2fd5 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:30.745517 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b360a-3744-49b2-8f5f-9c8f89163cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:30.746853 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b360a-3744-49b2-8f5f-9c8f89163cbf admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:30.747272 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b360a-3744-49b2-8f5f-9c8f89163cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:30.747672 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b360a-3744-49b2-8f5f-9c8f89163cbf admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:30.748415 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b360a-3744-49b2-8f5f-9c8f89163cbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:30.750818 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b360a-3744-49b2-8f5f-9c8f89163cbf admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:30.781212 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1127/2238] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:00:30 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:00:30.785544 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232bbf8c-5567-432e-b3f2-81eb5ebb18f6 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:30.796636 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232bbf8c-5567-432e-b3f2-81eb5ebb18f6 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:30.868124 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e12e78-d7b6-4c85-b82e-a82f305f7962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:30.868825 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e12e78-d7b6-4c85-b82e-a82f305f7962 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:30.869107 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e12e78-d7b6-4c85-b82e-a82f305f7962 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:30.869377 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e12e78-d7b6-4c85-b82e-a82f305f7962 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:30.870055 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e12e78-d7b6-4c85-b82e-a82f305f7962 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:30.872953 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e12e78-d7b6-4c85-b82e-a82f305f7962 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:30.894087 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1112/2239] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:00:30 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:00:30.898188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b360a-3744-49b2-8f5f-9c8f89163cbf admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:30.909235 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b360a-3744-49b2-8f5f-9c8f89163cbf admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:30.914176 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f6d734-22a6-4a20-9e11-32bf69b2c897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:30.914845 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f6d734-22a6-4a20-9e11-32bf69b2c897 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:30.915136 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f6d734-22a6-4a20-9e11-32bf69b2c897 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:30.915419 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f6d734-22a6-4a20-9e11-32bf69b2c897 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:30.916139 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f6d734-22a6-4a20-9e11-32bf69b2c897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:30.918564 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f6d734-22a6-4a20-9e11-32bf69b2c897 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:30.930121 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1128/2240] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:00:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:00:30.935659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e12e78-d7b6-4c85-b82e-a82f305f7962 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:30.946493 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e12e78-d7b6-4c85-b82e-a82f305f7962 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:30.951451 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa797e9d-acb9-41e4-89c7-c45102e0861c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:30.952116 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa797e9d-acb9-41e4-89c7-c45102e0861c admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:30.952400 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa797e9d-acb9-41e4-89c7-c45102e0861c admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:30.952711 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa797e9d-acb9-41e4-89c7-c45102e0861c admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:30.953435 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa797e9d-acb9-41e4-89c7-c45102e0861c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:30.955056 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa797e9d-acb9-41e4-89c7-c45102e0861c admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:30.965324 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1113/2241] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:00:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 06 15:00:30.969630 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f6d734-22a6-4a20-9e11-32bf69b2c897 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:30.979976 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f6d734-22a6-4a20-9e11-32bf69b2c897 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:30.985229 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cacc4d3-a973-434f-bbb3-31676ee9ce74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:30.986142 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cacc4d3-a973-434f-bbb3-31676ee9ce74 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:30.986426 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cacc4d3-a973-434f-bbb3-31676ee9ce74 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:30.986705 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cacc4d3-a973-434f-bbb3-31676ee9ce74 admin admin] PATH_INFO: `/v3/projects/c42bb11035094b63947e7651032038a4/users/a688eb03a17749d9a96463b27a51809c/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:31.002998 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cacc4d3-a973-434f-bbb3-31676ee9ce74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c42bb11035094b63947e7651032038a4, user_id=a688eb03a17749d9a96463b27a51809c, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:31.005709 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cacc4d3-a973-434f-bbb3-31676ee9ce74 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:31.019980 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1129/2242] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:00:30 2022] PUT /identity/v3/projects/c42bb11035094b63947e7651032038a4/users/a688eb03a17749d9a96463b27a51809c/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:00:31.032427 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a1e773-dae5-44c5-b767-c7226d0ea036 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:31.032833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a1e773-dae5-44c5-b767-c7226d0ea036 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:31.033112 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a1e773-dae5-44c5-b767-c7226d0ea036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:31.061016 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-78a1e773-dae5-44c5-b767-c7226d0ea036 None None] MFA Rules not processed for user `a688eb03a17749d9a96463b27a51809c`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:00:31.062226 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-78a1e773-dae5-44c5-b767-c7226d0ea036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:31.138516 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1114/2243] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:00:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:00:31.139366 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cacc4d3-a973-434f-bbb3-31676ee9ce74 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:31.212584 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cacc4d3-a973-434f-bbb3-31676ee9ce74 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:31.218832 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee84a1c-8595-43d0-b241-75a3dfa1491e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:31.219504 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee84a1c-8595-43d0-b241-75a3dfa1491e admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:31.219788 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee84a1c-8595-43d0-b241-75a3dfa1491e admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:31.220066 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee84a1c-8595-43d0-b241-75a3dfa1491e admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:31.220797 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee84a1c-8595-43d0-b241-75a3dfa1491e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:31.222709 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee84a1c-8595-43d0-b241-75a3dfa1491e admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:31.256284 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1130/2244] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:00:31 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:00:31.260418 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a1e773-dae5-44c5-b767-c7226d0ea036 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:31.271473 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a1e773-dae5-44c5-b767-c7226d0ea036 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:31.343154 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b91895a-5154-4225-8a64-90dba1fa7ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:31.344282 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91895a-5154-4225-8a64-90dba1fa7ae4 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:31.344823 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91895a-5154-4225-8a64-90dba1fa7ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:31.345214 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91895a-5154-4225-8a64-90dba1fa7ae4 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:31.345919 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b91895a-5154-4225-8a64-90dba1fa7ae4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:31.347842 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b91895a-5154-4225-8a64-90dba1fa7ae4 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:31.369908 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1115/2245] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:00:31 2022] POST /identity/v3/users => generated 298 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:00:31.375137 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee84a1c-8595-43d0-b241-75a3dfa1491e admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:31.393034 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee84a1c-8595-43d0-b241-75a3dfa1491e admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:31.397670 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677cac32-10ae-4e05-8ef6-c2eee4676dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:31.398338 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677cac32-10ae-4e05-8ef6-c2eee4676dd5 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:31.398626 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677cac32-10ae-4e05-8ef6-c2eee4676dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:31.398895 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677cac32-10ae-4e05-8ef6-c2eee4676dd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:31.399617 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677cac32-10ae-4e05-8ef6-c2eee4676dd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:31.401237 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677cac32-10ae-4e05-8ef6-c2eee4676dd5 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:31.415049 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1131/2246] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:00:31 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:00:31.420550 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b91895a-5154-4225-8a64-90dba1fa7ae4 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:31.431552 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b91895a-5154-4225-8a64-90dba1fa7ae4 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:31.438523 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5a048-a7f2-4e9e-84f4-7ad1d61088b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:31.439709 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5a048-a7f2-4e9e-84f4-7ad1d61088b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:31.440061 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5a048-a7f2-4e9e-84f4-7ad1d61088b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:31.440337 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5a048-a7f2-4e9e-84f4-7ad1d61088b2 admin admin] PATH_INFO: `/v3/projects/e849b9ff5a9548cf9013033f1ffab879/users/dbea0435e5bd479f85e413a80437bc6b/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:31.458243 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a5a048-a7f2-4e9e-84f4-7ad1d61088b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e849b9ff5a9548cf9013033f1ffab879, user_id=dbea0435e5bd479f85e413a80437bc6b, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:31.461003 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a5a048-a7f2-4e9e-84f4-7ad1d61088b2 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:31.479521 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1116/2247] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:00:31 2022] PUT /identity/v3/projects/e849b9ff5a9548cf9013033f1ffab879/users/dbea0435e5bd479f85e413a80437bc6b/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:00:31.483847 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677cac32-10ae-4e05-8ef6-c2eee4676dd5 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:31.493885 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677cac32-10ae-4e05-8ef6-c2eee4676dd5 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:31.562258 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f72fb-5f41-4b03-a4c8-68768ebb3068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:31.562928 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f72fb-5f41-4b03-a4c8-68768ebb3068 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:31.563209 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f72fb-5f41-4b03-a4c8-68768ebb3068 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:31.563482 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f72fb-5f41-4b03-a4c8-68768ebb3068 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:31.564206 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f72fb-5f41-4b03-a4c8-68768ebb3068 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:31.566188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f72fb-5f41-4b03-a4c8-68768ebb3068 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:31.577004 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1132/2248] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:00:31 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:00:31.581894 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5a048-a7f2-4e9e-84f4-7ad1d61088b2 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:31.592813 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5a048-a7f2-4e9e-84f4-7ad1d61088b2 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:31.602179 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7007a414-7fe7-41e5-a059-e7b6d0e9adcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:31.603817 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007a414-7fe7-41e5-a059-e7b6d0e9adcd admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:31.603817 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007a414-7fe7-41e5-a059-e7b6d0e9adcd admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:31.603817 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007a414-7fe7-41e5-a059-e7b6d0e9adcd admin admin] PATH_INFO: `/v3/projects/e849b9ff5a9548cf9013033f1ffab879/users/dbea0435e5bd479f85e413a80437bc6b/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:31.606307 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7007a414-7fe7-41e5-a059-e7b6d0e9adcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e849b9ff5a9548cf9013033f1ffab879, user_id=dbea0435e5bd479f85e413a80437bc6b, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:31.608978 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7007a414-7fe7-41e5-a059-e7b6d0e9adcd admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:31.619434 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1117/2249] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:00:31 2022] PUT /identity/v3/projects/e849b9ff5a9548cf9013033f1ffab879/users/dbea0435e5bd479f85e413a80437bc6b/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:00:31.631064 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08698f6c-9889-4f05-bfd0-d4c8008c28c8 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:31.631357 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08698f6c-9889-4f05-bfd0-d4c8008c28c8 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:31.631646 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08698f6c-9889-4f05-bfd0-d4c8008c28c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:31.659052 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-08698f6c-9889-4f05-bfd0-d4c8008c28c8 None None] MFA Rules not processed for user `dbea0435e5bd479f85e413a80437bc6b`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:00:31.660182 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-08698f6c-9889-4f05-bfd0-d4c8008c28c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:31.731916 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1133/2250] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:00:31 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:00:31.743889 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7007a414-7fe7-41e5-a059-e7b6d0e9adcd admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:31.745640 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-7007a414-7fe7-41e5-a059-e7b6d0e9adcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:31.814690 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7007a414-7fe7-41e5-a059-e7b6d0e9adcd admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:31.818406 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c4df2f-407d-49e6-bebe-1a0db3f34d52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:31.819082 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c4df2f-407d-49e6-bebe-1a0db3f34d52 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:31.819347 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c4df2f-407d-49e6-bebe-1a0db3f34d52 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:31.819622 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c4df2f-407d-49e6-bebe-1a0db3f34d52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:31.879034 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c4df2f-407d-49e6-bebe-1a0db3f34d52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:31.880510 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c4df2f-407d-49e6-bebe-1a0db3f34d52 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:31.889689 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1118/2251] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:00:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 15:00:32.083929 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08698f6c-9889-4f05-bfd0-d4c8008c28c8 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:32.085831 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-08698f6c-9889-4f05-bfd0-d4c8008c28c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:32.166933 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08698f6c-9889-4f05-bfd0-d4c8008c28c8 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:32.171328 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da219afc-8695-4464-858a-87b5140159ac service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9be93f9ff7e46babc8ee87b50b9c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:32.171954 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da219afc-8695-4464-858a-87b5140159ac service glance] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:32.172237 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da219afc-8695-4464-858a-87b5140159ac service glance] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:32.172642 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da219afc-8695-4464-858a-87b5140159ac service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:32.174676 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-da219afc-8695-4464-858a-87b5140159ac service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:32.244816 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da219afc-8695-4464-858a-87b5140159ac service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:32.247463 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da219afc-8695-4464-858a-87b5140159ac service glance] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:32.258083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1134/2252] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:00:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:00:33.459495 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c4df2f-407d-49e6-bebe-1a0db3f34d52 service cinder] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:33.460989 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b2c4df2f-407d-49e6-bebe-1a0db3f34d52 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:33.472838 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c4df2f-407d-49e6-bebe-1a0db3f34d52 service cinder] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:33.476648 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285a824a-6a8f-44be-932e-e38ecdb3e2fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:33.477354 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a824a-6a8f-44be-932e-e38ecdb3e2fe service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:33.477630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a824a-6a8f-44be-932e-e38ecdb3e2fe service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:33.477900 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a824a-6a8f-44be-932e-e38ecdb3e2fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:33.556081 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285a824a-6a8f-44be-932e-e38ecdb3e2fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:33.559140 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285a824a-6a8f-44be-932e-e38ecdb3e2fe service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:33.576097 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1119/2253] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:00:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:00:36.786827 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da219afc-8695-4464-858a-87b5140159ac service glance] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:36.797283 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da219afc-8695-4464-858a-87b5140159ac service glance] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:36.800930 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9d6809-03f2-4943-99d1-8d65d94642d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:36.801576 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d6809-03f2-4943-99d1-8d65d94642d1 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:36.801853 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d6809-03f2-4943-99d1-8d65d94642d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:36.802121 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d6809-03f2-4943-99d1-8d65d94642d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:36.812793 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9d6809-03f2-4943-99d1-8d65d94642d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:36.814219 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9d6809-03f2-4943-99d1-8d65d94642d1 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:36.823726 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1135/2254] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:00:36 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:00:40.670879 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285a824a-6a8f-44be-932e-e38ecdb3e2fe service cinder] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:40.734124 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285a824a-6a8f-44be-932e-e38ecdb3e2fe service cinder] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:40.738698 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19ea10-d2d5-43a2-9e8b-fe146daac37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:40.739583 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19ea10-d2d5-43a2-9e8b-fe146daac37f admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:40.739864 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19ea10-d2d5-43a2-9e8b-fe146daac37f admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:40.740143 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19ea10-d2d5-43a2-9e8b-fe146daac37f admin admin] PATH_INFO: `/v3/users/c4152e495e9a4d9d8b25e74769781afb` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:40.741359 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a19ea10-d2d5-43a2-9e8b-fe146daac37f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4152e495e9a4d9d8b25e74769781afb)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:40.743294 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a19ea10-d2d5-43a2-9e8b-fe146daac37f admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:40.794022 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-5a19ea10-d2d5-43a2-9e8b-fe146daac37f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4152e495e9a4d9d8b25e74769781afb'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:40.794593 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-5a19ea10-d2d5-43a2-9e8b-fe146daac37f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4152e495e9a4d9d8b25e74769781afb'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:40.803093 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-5a19ea10-d2d5-43a2-9e8b-fe146daac37f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4152e495e9a4d9d8b25e74769781afb'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:40.810815 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-5a19ea10-d2d5-43a2-9e8b-fe146daac37f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4152e495e9a4d9d8b25e74769781afb'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:40.811514 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-5a19ea10-d2d5-43a2-9e8b-fe146daac37f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4152e495e9a4d9d8b25e74769781afb'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:40.829592 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1120/2255] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:00:40 2022] DELETE /identity/v3/users/c4152e495e9a4d9d8b25e74769781afb => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:00:40.834290 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c7f327-ccd7-4148-a644-c76e682fc5c5 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:40.834760 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c7f327-ccd7-4148-a644-c76e682fc5c5 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:40.834973 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c7f327-ccd7-4148-a644-c76e682fc5c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:40.852142 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-74c7f327-ccd7-4148-a644-c76e682fc5c5 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:00:40.853519 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-74c7f327-ccd7-4148-a644-c76e682fc5c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:40.917776 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1136/2256] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:00:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:00:40.927676 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19ea10-d2d5-43a2-9e8b-fe146daac37f admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:40.929095 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-5a19ea10-d2d5-43a2-9e8b-fe146daac37f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:40.993280 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19ea10-d2d5-43a2-9e8b-fe146daac37f admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:40.998115 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f5c74-43cd-4f1c-8683-413af0a62a3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:40.998760 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f5c74-43cd-4f1c-8683-413af0a62a3f service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:40.999040 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f5c74-43cd-4f1c-8683-413af0a62a3f service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:40.999315 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f5c74-43cd-4f1c-8683-413af0a62a3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:41.008761 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661f5c74-43cd-4f1c-8683-413af0a62a3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:41.010188 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661f5c74-43cd-4f1c-8683-413af0a62a3f service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:41.020862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1121/2257] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:00:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:00:41.379802 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c7f327-ccd7-4148-a644-c76e682fc5c5 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:41.381695 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-74c7f327-ccd7-4148-a644-c76e682fc5c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:41.423495 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c7f327-ccd7-4148-a644-c76e682fc5c5 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:41.434937 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a24b57-bf17-4f57-a5aa-12d885d8e8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:41.437649 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a24b57-bf17-4f57-a5aa-12d885d8e8f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:41.438500 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a24b57-bf17-4f57-a5aa-12d885d8e8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:41.439135 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a24b57-bf17-4f57-a5aa-12d885d8e8f9 admin admin] PATH_INFO: `/v3/projects/fde1b936bf84416aa4c1f654d295de0f` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:41.441194 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a24b57-bf17-4f57-a5aa-12d885d8e8f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fde1b936bf84416aa4c1f654d295de0f)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:41.443248 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a24b57-bf17-4f57-a5aa-12d885d8e8f9 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:41.508696 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-06a24b57-bf17-4f57-a5aa-12d885d8e8f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fde1b936bf84416aa4c1f654d295de0f', 'request_id': 'req-06a24b57-bf17-4f57-a5aa-12d885d8e8f9'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:41.516474 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-06a24b57-bf17-4f57-a5aa-12d885d8e8f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fde1b936bf84416aa4c1f654d295de0f', 'request_id': 'req-06a24b57-bf17-4f57-a5aa-12d885d8e8f9'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:41.526299 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-06a24b57-bf17-4f57-a5aa-12d885d8e8f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fde1b936bf84416aa4c1f654d295de0f', 'request_id': 'req-06a24b57-bf17-4f57-a5aa-12d885d8e8f9'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:41.545748 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-06a24b57-bf17-4f57-a5aa-12d885d8e8f9 admin admin] The token cache is being invalidate because project fde1b936bf84416aa4c1f654d295de0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:00:41.546121 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-06a24b57-bf17-4f57-a5aa-12d885d8e8f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:41.549821 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-06a24b57-bf17-4f57-a5aa-12d885d8e8f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:41.551999 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1137/2258] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:00:41 2022] DELETE /identity/v3/projects/fde1b936bf84416aa4c1f654d295de0f => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:00:41.556867 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f5c74-43cd-4f1c-8683-413af0a62a3f service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:41.576307 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f5c74-43cd-4f1c-8683-413af0a62a3f service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:41.635730 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab165c96-161b-4243-9dcc-c416999a327d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:41.636650 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab165c96-161b-4243-9dcc-c416999a327d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:41.637809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab165c96-161b-4243-9dcc-c416999a327d admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:41.638297 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab165c96-161b-4243-9dcc-c416999a327d admin admin] PATH_INFO: `/v3/users/a688eb03a17749d9a96463b27a51809c` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:41.639758 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab165c96-161b-4243-9dcc-c416999a327d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a688eb03a17749d9a96463b27a51809c)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:41.641713 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab165c96-161b-4243-9dcc-c416999a327d admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:41.691222 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-ab165c96-161b-4243-9dcc-c416999a327d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a688eb03a17749d9a96463b27a51809c'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:41.697410 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-ab165c96-161b-4243-9dcc-c416999a327d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a688eb03a17749d9a96463b27a51809c'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:41.708934 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-ab165c96-161b-4243-9dcc-c416999a327d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a688eb03a17749d9a96463b27a51809c'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:41.718304 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-ab165c96-161b-4243-9dcc-c416999a327d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a688eb03a17749d9a96463b27a51809c'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:41.719023 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-ab165c96-161b-4243-9dcc-c416999a327d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a688eb03a17749d9a96463b27a51809c'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:41.737332 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1122/2259] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:00:41 2022] DELETE /identity/v3/users/a688eb03a17749d9a96463b27a51809c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:00:42.029107 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a24b57-bf17-4f57-a5aa-12d885d8e8f9 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:42.030881 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-06a24b57-bf17-4f57-a5aa-12d885d8e8f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:42.043983 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a24b57-bf17-4f57-a5aa-12d885d8e8f9 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:42.102689 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7b0bc-8026-4e37-b09d-4875c113eafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:42.103565 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7b0bc-8026-4e37-b09d-4875c113eafc admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:42.103878 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7b0bc-8026-4e37-b09d-4875c113eafc admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:42.104158 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7b0bc-8026-4e37-b09d-4875c113eafc admin admin] PATH_INFO: `/v3/projects/c42bb11035094b63947e7651032038a4` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:42.105377 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7b0bc-8026-4e37-b09d-4875c113eafc admin admin] RBAC: Authorizing `identity:delete_project(project_id=c42bb11035094b63947e7651032038a4)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:42.107520 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7b0bc-8026-4e37-b09d-4875c113eafc admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:42.158437 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ceb7b0bc-8026-4e37-b09d-4875c113eafc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c42bb11035094b63947e7651032038a4', 'request_id': 'req-ceb7b0bc-8026-4e37-b09d-4875c113eafc'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:42.164218 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ceb7b0bc-8026-4e37-b09d-4875c113eafc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c42bb11035094b63947e7651032038a4', 'request_id': 'req-ceb7b0bc-8026-4e37-b09d-4875c113eafc'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:42.172178 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ceb7b0bc-8026-4e37-b09d-4875c113eafc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c42bb11035094b63947e7651032038a4', 'request_id': 'req-ceb7b0bc-8026-4e37-b09d-4875c113eafc'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:42.183584 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ceb7b0bc-8026-4e37-b09d-4875c113eafc admin admin] The token cache is being invalidate because project c42bb11035094b63947e7651032038a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:00:42.183877 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ceb7b0bc-8026-4e37-b09d-4875c113eafc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:42.185100 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ceb7b0bc-8026-4e37-b09d-4875c113eafc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:42.186638 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1138/2260] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:00:42 2022] DELETE /identity/v3/projects/c42bb11035094b63947e7651032038a4 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:00:42.190743 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab165c96-161b-4243-9dcc-c416999a327d admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:42.192193 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ab165c96-161b-4243-9dcc-c416999a327d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:42.205863 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab165c96-161b-4243-9dcc-c416999a327d admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:42.263303 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1bcd24-0d73-4c5d-9857-f6eecab45b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:42.264313 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1bcd24-0d73-4c5d-9857-f6eecab45b40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:42.264627 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1bcd24-0d73-4c5d-9857-f6eecab45b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:42.264928 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1bcd24-0d73-4c5d-9857-f6eecab45b40 admin admin] PATH_INFO: `/v3/users/dbea0435e5bd479f85e413a80437bc6b` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:42.266235 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1bcd24-0d73-4c5d-9857-f6eecab45b40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbea0435e5bd479f85e413a80437bc6b)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:42.268247 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1bcd24-0d73-4c5d-9857-f6eecab45b40 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:42.317248 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-7f1bcd24-0d73-4c5d-9857-f6eecab45b40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbea0435e5bd479f85e413a80437bc6b'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:42.317952 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-7f1bcd24-0d73-4c5d-9857-f6eecab45b40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbea0435e5bd479f85e413a80437bc6b'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:42.327197 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-7f1bcd24-0d73-4c5d-9857-f6eecab45b40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbea0435e5bd479f85e413a80437bc6b'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:42.334456 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-7f1bcd24-0d73-4c5d-9857-f6eecab45b40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbea0435e5bd479f85e413a80437bc6b'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:42.335225 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-7f1bcd24-0d73-4c5d-9857-f6eecab45b40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbea0435e5bd479f85e413a80437bc6b'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:42.353281 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1123/2261] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:00:42 2022] DELETE /identity/v3/users/dbea0435e5bd479f85e413a80437bc6b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 15:00:42.643472 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7b0bc-8026-4e37-b09d-4875c113eafc admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:42.644969 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-ceb7b0bc-8026-4e37-b09d-4875c113eafc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:42.659686 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7b0bc-8026-4e37-b09d-4875c113eafc admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:42.735970 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4751c-dc92-44d7-a41d-cd7305492059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:42.737067 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4751c-dc92-44d7-a41d-cd7305492059 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:42.737357 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4751c-dc92-44d7-a41d-cd7305492059 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:42.737636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4751c-dc92-44d7-a41d-cd7305492059 admin admin] PATH_INFO: `/v3/projects/e849b9ff5a9548cf9013033f1ffab879` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:42.738857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c4751c-dc92-44d7-a41d-cd7305492059 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e849b9ff5a9548cf9013033f1ffab879)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:42.740892 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c4751c-dc92-44d7-a41d-cd7305492059 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:42.802724 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d2c4751c-dc92-44d7-a41d-cd7305492059 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e849b9ff5a9548cf9013033f1ffab879', 'request_id': 'req-d2c4751c-dc92-44d7-a41d-cd7305492059'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:42.811233 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d2c4751c-dc92-44d7-a41d-cd7305492059 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e849b9ff5a9548cf9013033f1ffab879', 'request_id': 'req-d2c4751c-dc92-44d7-a41d-cd7305492059'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:42.820065 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d2c4751c-dc92-44d7-a41d-cd7305492059 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e849b9ff5a9548cf9013033f1ffab879', 'request_id': 'req-d2c4751c-dc92-44d7-a41d-cd7305492059'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:42.834837 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d2c4751c-dc92-44d7-a41d-cd7305492059 admin admin] The token cache is being invalidate because project e849b9ff5a9548cf9013033f1ffab879 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:00:42.835239 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d2c4751c-dc92-44d7-a41d-cd7305492059 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:42.835965 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d2c4751c-dc92-44d7-a41d-cd7305492059 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:00:42.837641 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1139/2262] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:00:42 2022] DELETE /identity/v3/projects/e849b9ff5a9548cf9013033f1ffab879 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:00:42.843200 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32aad80-cd40-493b-8664-abf5a09a9995 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:42.843773 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32aad80-cd40-493b-8664-abf5a09a9995 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:42.843773 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32aad80-cd40-493b-8664-abf5a09a9995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:42.864445 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-a32aad80-cd40-493b-8664-abf5a09a9995 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:00:42.865983 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-a32aad80-cd40-493b-8664-abf5a09a9995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:42.933138 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1124/2263] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:00:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:00:42.941843 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a3622-a86b-4227-9915-3666192b5ead None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:42.942128 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a3622-a86b-4227-9915-3666192b5ead None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:42.942404 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a3622-a86b-4227-9915-3666192b5ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:42.964841 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-db7a3622-a86b-4227-9915-3666192b5ead None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:00:42.965713 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-db7a3622-a86b-4227-9915-3666192b5ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:42.974832 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1140/2264] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:00:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:00:42.978355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32aad80-cd40-493b-8664-abf5a09a9995 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:42.992850 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32aad80-cd40-493b-8664-abf5a09a9995 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:42.995038 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c77b25d-9a6d-4050-a94a-e5a190a5a7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:42.995708 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77b25d-9a6d-4050-a94a-e5a190a5a7be admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:42.995994 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77b25d-9a6d-4050-a94a-e5a190a5a7be admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:42.996295 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77b25d-9a6d-4050-a94a-e5a190a5a7be admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:42.997597 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c77b25d-9a6d-4050-a94a-e5a190a5a7be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 15:00:42.997908 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c77b25d-9a6d-4050-a94a-e5a190a5a7be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:42.999629 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c77b25d-9a6d-4050-a94a-e5a190a5a7be admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:43.014690 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1125/2265] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 15:00:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 15:00:43.019259 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a3622-a86b-4227-9915-3666192b5ead None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:43.030598 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a3622-a86b-4227-9915-3666192b5ead None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:43.036332 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f4a2b-5808-45d0-bb97-32c085ee0a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:43.037048 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f4a2b-5808-45d0-bb97-32c085ee0a2f admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:43.037341 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f4a2b-5808-45d0-bb97-32c085ee0a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:43.037636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f4a2b-5808-45d0-bb97-32c085ee0a2f admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:43.038464 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f4a2b-5808-45d0-bb97-32c085ee0a2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:43.040551 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f4a2b-5808-45d0-bb97-32c085ee0a2f admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:43.068084 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1141/2266] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:00:43 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:00:43.072183 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c77b25d-9a6d-4050-a94a-e5a190a5a7be admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:43.082649 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c77b25d-9a6d-4050-a94a-e5a190a5a7be admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:43.157954 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea593c0-c63b-4d9a-8a3b-0068f0715bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:43.158940 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea593c0-c63b-4d9a-8a3b-0068f0715bab admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:43.158940 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea593c0-c63b-4d9a-8a3b-0068f0715bab admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:43.159171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea593c0-c63b-4d9a-8a3b-0068f0715bab admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:43.159875 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea593c0-c63b-4d9a-8a3b-0068f0715bab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:43.161852 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea593c0-c63b-4d9a-8a3b-0068f0715bab admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:43.187100 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1126/2267] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:00:43 2022] POST /identity/v3/users => generated 291 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:00:43.188378 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f4a2b-5808-45d0-bb97-32c085ee0a2f admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:43.199013 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f4a2b-5808-45d0-bb97-32c085ee0a2f admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:43.204371 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41f4942-29ae-4939-9a86-1ad349165706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:43.205404 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f4942-29ae-4939-9a86-1ad349165706 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:43.205708 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f4942-29ae-4939-9a86-1ad349165706 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:43.206005 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f4942-29ae-4939-9a86-1ad349165706 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:43.206722 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41f4942-29ae-4939-9a86-1ad349165706 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:43.208309 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41f4942-29ae-4939-9a86-1ad349165706 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:43.222835 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1142/2268] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:00:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:00:43.223996 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea593c0-c63b-4d9a-8a3b-0068f0715bab admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:43.235198 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea593c0-c63b-4d9a-8a3b-0068f0715bab admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:43.239957 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f3607-e5d4-4182-a7b0-3bd81938c5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:43.240681 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f3607-e5d4-4182-a7b0-3bd81938c5ce admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:43.240963 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f3607-e5d4-4182-a7b0-3bd81938c5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:43.241246 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f3607-e5d4-4182-a7b0-3bd81938c5ce admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:43.242006 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484f3607-e5d4-4182-a7b0-3bd81938c5ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:43.243627 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484f3607-e5d4-4182-a7b0-3bd81938c5ce admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:43.257530 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1127/2269] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:00:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 06 15:00:43.265869 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41f4942-29ae-4939-9a86-1ad349165706 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:43.277150 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41f4942-29ae-4939-9a86-1ad349165706 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:43.283324 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5d437e-e56c-49c3-9a84-0160c64ca95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:43.284253 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d437e-e56c-49c3-9a84-0160c64ca95d admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:43.284536 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d437e-e56c-49c3-9a84-0160c64ca95d admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:43.284853 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d437e-e56c-49c3-9a84-0160c64ca95d admin admin] PATH_INFO: `/v3/projects/d266c6c15e474b7885134dbe78a29833/users/db19f328e0e448eba77532cdd7bb4c78/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:43.308564 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5d437e-e56c-49c3-9a84-0160c64ca95d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d266c6c15e474b7885134dbe78a29833, user_id=db19f328e0e448eba77532cdd7bb4c78, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:43.311905 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5d437e-e56c-49c3-9a84-0160c64ca95d admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:43.329211 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1143/2270] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:00:43 2022] PUT /identity/v3/projects/d266c6c15e474b7885134dbe78a29833/users/db19f328e0e448eba77532cdd7bb4c78/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:00:43.334920 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b4798-940c-4434-8efe-eee70cdc81c2 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:43.335193 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b4798-940c-4434-8efe-eee70cdc81c2 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:43.335477 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b4798-940c-4434-8efe-eee70cdc81c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:43.353575 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-3d8b4798-940c-4434-8efe-eee70cdc81c2 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:00:43.354789 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-3d8b4798-940c-4434-8efe-eee70cdc81c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:43.420325 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1128/2271] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:00:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:00:43.430237 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5d437e-e56c-49c3-9a84-0160c64ca95d admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:43.432448 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-6b5d437e-e56c-49c3-9a84-0160c64ca95d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:43.498266 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5d437e-e56c-49c3-9a84-0160c64ca95d admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:43.502002 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0aae89-f0b8-4e1b-9f32-29b2bff02939 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:43.502639 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0aae89-f0b8-4e1b-9f32-29b2bff02939 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:43.502918 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0aae89-f0b8-4e1b-9f32-29b2bff02939 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:43.503191 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0aae89-f0b8-4e1b-9f32-29b2bff02939 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:43.515030 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0aae89-f0b8-4e1b-9f32-29b2bff02939 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:43.516748 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0aae89-f0b8-4e1b-9f32-29b2bff02939 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:43.529161 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1144/2272] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:00:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:00:43.961007 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778c66d-6cc1-435c-a070-fbc6f967926b None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:43.961905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778c66d-6cc1-435c-a070-fbc6f967926b None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:43.962219 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778c66d-6cc1-435c-a070-fbc6f967926b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:43.998750 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-d778c66d-6cc1-435c-a070-fbc6f967926b None None] MFA Rules not processed for user `a7dc1bc53c59423e934f7831ea022c73`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:00:43.999995 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d778c66d-6cc1-435c-a070-fbc6f967926b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:44.079335 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1129/2273] 10.222.0.23 () {58 vars in 1009 bytes} [Fri May 6 15:00:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:00:44.137162 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0aae89-f0b8-4e1b-9f32-29b2bff02939 service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:44.139507 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-6b0aae89-f0b8-4e1b-9f32-29b2bff02939 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:44.243403 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0aae89-f0b8-4e1b-9f32-29b2bff02939 service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:44.249287 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee97e44-e8d2-40d1-b06c-b4df52f66a76 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf9eed62c8f1409282c2af6488426a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:44.249990 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee97e44-e8d2-40d1-b06c-b4df52f66a76 service swift] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:44.250278 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee97e44-e8d2-40d1-b06c-b4df52f66a76 service swift] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:44.250561 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee97e44-e8d2-40d1-b06c-b4df52f66a76 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:44.265391 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee97e44-e8d2-40d1-b06c-b4df52f66a76 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:44.265391 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee97e44-e8d2-40d1-b06c-b4df52f66a76 service swift] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:44.268508 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554bc65c-ced4-401a-b97e-b160e2782bf2 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:44.268833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554bc65c-ced4-401a-b97e-b160e2782bf2 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:44.269419 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554bc65c-ced4-401a-b97e-b160e2782bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:44.274901 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1145/2274] 10.222.0.23 () {60 vars in 1384 bytes} [Fri May 6 15:00:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 06 15:00:44.328796 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-554bc65c-ced4-401a-b97e-b160e2782bf2 None None] MFA Rules not processed for user `db19f328e0e448eba77532cdd7bb4c78`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:00:44.330723 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-554bc65c-ced4-401a-b97e-b160e2782bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:44.409503 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1130/2275] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:00:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 15:00:44.420928 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee97e44-e8d2-40d1-b06c-b4df52f66a76 service swift] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:00:44.422922 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-7ee97e44-e8d2-40d1-b06c-b4df52f66a76 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:44.508260 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee97e44-e8d2-40d1-b06c-b4df52f66a76 service swift] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:00:44.513047 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef1b68-4b7e-4968-9260-52ff7b665bd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:00:44.513851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef1b68-4b7e-4968-9260-52ff7b665bd0 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:44.514250 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef1b68-4b7e-4968-9260-52ff7b665bd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:44.514642 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef1b68-4b7e-4968-9260-52ff7b665bd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:44.527696 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef1b68-4b7e-4968-9260-52ff7b665bd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:00:44.530126 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef1b68-4b7e-4968-9260-52ff7b665bd0 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:00:44.540550 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1146/2276] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:00:44 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 15:00:46.014656 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97212eaf-14ad-4617-a31c-8fc367dfee9d None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:00:46.015267 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97212eaf-14ad-4617-a31c-8fc367dfee9d None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:00:46.015439 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97212eaf-14ad-4617-a31c-8fc367dfee9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:00:46.039734 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-97212eaf-14ad-4617-a31c-8fc367dfee9d None None] MFA Rules not processed for user `db19f328e0e448eba77532cdd7bb4c78`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:00:46.041658 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-97212eaf-14ad-4617-a31c-8fc367dfee9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:00:46.057867 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1131/2277] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:00:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:01:12.528770 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31fb69-25b3-47ba-acc8-9fcc685c8c17 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:12.529449 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31fb69-25b3-47ba-acc8-9fcc685c8c17 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:12.529449 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31fb69-25b3-47ba-acc8-9fcc685c8c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:12.546273 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-de31fb69-25b3-47ba-acc8-9fcc685c8c17 None None] MFA Rules not processed for user `4b3af1d82c33446e998f07bce54ef941`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:01:12.547580 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-de31fb69-25b3-47ba-acc8-9fcc685c8c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:01:12.604479 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1147/2278] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:01:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:01:22.326261 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97212eaf-14ad-4617-a31c-8fc367dfee9d None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:01:22.328579 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-97212eaf-14ad-4617-a31c-8fc367dfee9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:01:22.345912 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97212eaf-14ad-4617-a31c-8fc367dfee9d None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:01:22.354444 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed0e0f-7dfb-418b-a7f3-9b9ea9b05272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:01:22.356360 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed0e0f-7dfb-418b-a7f3-9b9ea9b05272 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:22.356915 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed0e0f-7dfb-418b-a7f3-9b9ea9b05272 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:22.358389 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed0e0f-7dfb-418b-a7f3-9b9ea9b05272 admin admin] PATH_INFO: `/v3/users/88e831bc4ff040f98cb2431f02426312` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:22.361321 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed0e0f-7dfb-418b-a7f3-9b9ea9b05272 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88e831bc4ff040f98cb2431f02426312)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:01:22.363898 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed0e0f-7dfb-418b-a7f3-9b9ea9b05272 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:01:22.443688 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-eeed0e0f-7dfb-418b-a7f3-9b9ea9b05272 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88e831bc4ff040f98cb2431f02426312'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:01:22.444804 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-eeed0e0f-7dfb-418b-a7f3-9b9ea9b05272 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88e831bc4ff040f98cb2431f02426312'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:01:22.463717 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-eeed0e0f-7dfb-418b-a7f3-9b9ea9b05272 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88e831bc4ff040f98cb2431f02426312'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:01:22.477559 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-eeed0e0f-7dfb-418b-a7f3-9b9ea9b05272 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88e831bc4ff040f98cb2431f02426312'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:01:22.478388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-eeed0e0f-7dfb-418b-a7f3-9b9ea9b05272 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88e831bc4ff040f98cb2431f02426312'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:01:22.507037 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1132/2279] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:01:22 2022] DELETE /identity/v3/users/88e831bc4ff040f98cb2431f02426312 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 15:01:22.512063 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7f8de-a512-4811-837f-2d21b2a57fba None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:22.513035 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7f8de-a512-4811-837f-2d21b2a57fba None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:22.513397 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7f8de-a512-4811-837f-2d21b2a57fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:22.558828 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-e4a7f8de-a512-4811-837f-2d21b2a57fba None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:01:22.563011 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e4a7f8de-a512-4811-837f-2d21b2a57fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:01:22.680263 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1148/2280] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:01:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:01:22.692745 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed0e0f-7dfb-418b-a7f3-9b9ea9b05272 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:01:22.694484 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-eeed0e0f-7dfb-418b-a7f3-9b9ea9b05272 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:01:22.829210 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed0e0f-7dfb-418b-a7f3-9b9ea9b05272 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:01:22.834200 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467f029f-028d-4162-b35d-df02435414c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:01:22.835010 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f029f-028d-4162-b35d-df02435414c3 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:22.835304 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f029f-028d-4162-b35d-df02435414c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:22.835583 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f029f-028d-4162-b35d-df02435414c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:22.848117 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467f029f-028d-4162-b35d-df02435414c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:01:22.849901 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467f029f-028d-4162-b35d-df02435414c3 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:01:22.865478 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1133/2281] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:01:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:01:23.037651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8adb5ec-f35f-4a39-ae0a-27610d4adea0 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:23.038132 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8adb5ec-f35f-4a39-ae0a-27610d4adea0 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:23.038418 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8adb5ec-f35f-4a39-ae0a-27610d4adea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:23.118530 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-e8adb5ec-f35f-4a39-ae0a-27610d4adea0 None None] MFA Rules not processed for user `a7dc1bc53c59423e934f7831ea022c73`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:01:23.119805 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e8adb5ec-f35f-4a39-ae0a-27610d4adea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:01:23.192392 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1149/2282] 10.222.0.23 () {58 vars in 967 bytes} [Fri May 6 15:01:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:01:23.206555 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467f029f-028d-4162-b35d-df02435414c3 service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:01:23.208046 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-467f029f-028d-4162-b35d-df02435414c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:01:23.242438 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8adb5ec-f35f-4a39-ae0a-27610d4adea0 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:01:23.243829 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e8adb5ec-f35f-4a39-ae0a-27610d4adea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:01:23.261757 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8adb5ec-f35f-4a39-ae0a-27610d4adea0 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:01:23.275579 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab4330a-c0d1-43a3-abb0-d4a73446447e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:01:23.276459 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4330a-c0d1-43a3-abb0-d4a73446447e admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:23.277827 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4330a-c0d1-43a3-abb0-d4a73446447e admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:23.278168 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4330a-c0d1-43a3-abb0-d4a73446447e admin admin] PATH_INFO: `/v3/projects/4c4671efc7be4313a67a83fbf93234fc` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:23.279343 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab4330a-c0d1-43a3-abb0-d4a73446447e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c4671efc7be4313a67a83fbf93234fc)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:01:23.281788 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab4330a-c0d1-43a3-abb0-d4a73446447e admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:01:23.321449 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467f029f-028d-4162-b35d-df02435414c3 service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:01:23.327276 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69797166-f5e0-47be-a68d-2b95d7da0bb4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf9eed62c8f1409282c2af6488426a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:01:23.328098 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69797166-f5e0-47be-a68d-2b95d7da0bb4 service swift] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:23.328450 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69797166-f5e0-47be-a68d-2b95d7da0bb4 service swift] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:23.328819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69797166-f5e0-47be-a68d-2b95d7da0bb4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:23.342334 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69797166-f5e0-47be-a68d-2b95d7da0bb4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:01:23.344087 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69797166-f5e0-47be-a68d-2b95d7da0bb4 service swift] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:01:23.346744 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-3ab4330a-c0d1-43a3-abb0-d4a73446447e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c4671efc7be4313a67a83fbf93234fc', 'request_id': 'req-3ab4330a-c0d1-43a3-abb0-d4a73446447e'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:01:23.352210 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-3ab4330a-c0d1-43a3-abb0-d4a73446447e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c4671efc7be4313a67a83fbf93234fc', 'request_id': 'req-3ab4330a-c0d1-43a3-abb0-d4a73446447e'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:01:23.353716 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1134/2283] 10.222.0.23 () {60 vars in 1384 bytes} [Fri May 6 15:01:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 06 15:01:23.369131 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-3ab4330a-c0d1-43a3-abb0-d4a73446447e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c4671efc7be4313a67a83fbf93234fc', 'request_id': 'req-3ab4330a-c0d1-43a3-abb0-d4a73446447e'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:01:23.381264 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-3ab4330a-c0d1-43a3-abb0-d4a73446447e admin admin] The token cache is being invalidate because project 4c4671efc7be4313a67a83fbf93234fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:01:23.381607 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-3ab4330a-c0d1-43a3-abb0-d4a73446447e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:01:23.382287 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-3ab4330a-c0d1-43a3-abb0-d4a73446447e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:01:23.384163 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1150/2284] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:01:23 2022] DELETE /identity/v3/projects/4c4671efc7be4313a67a83fbf93234fc => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 15:01:24.832223 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69797166-f5e0-47be-a68d-2b95d7da0bb4 service swift] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:01:24.835473 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-69797166-f5e0-47be-a68d-2b95d7da0bb4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:01:24.960330 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69797166-f5e0-47be-a68d-2b95d7da0bb4 service swift] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:01:25.040548 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a139d76-1ac2-4a0a-90d8-01644b33883e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887a619ef5b5417db98ff15a9f712724', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:01:25.041245 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a139d76-1ac2-4a0a-90d8-01644b33883e service placement] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:25.041531 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a139d76-1ac2-4a0a-90d8-01644b33883e service placement] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:25.041808 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a139d76-1ac2-4a0a-90d8-01644b33883e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:25.044363 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-9a139d76-1ac2-4a0a-90d8-01644b33883e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:01:25.060453 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a139d76-1ac2-4a0a-90d8-01644b33883e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:01:25.062507 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a139d76-1ac2-4a0a-90d8-01644b33883e service placement] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:01:25.078032 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1135/2285] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:01:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 15:01:25.629508 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab4330a-c0d1-43a3-abb0-d4a73446447e admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:01:25.631939 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-3ab4330a-c0d1-43a3-abb0-d4a73446447e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:01:25.704883 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab4330a-c0d1-43a3-abb0-d4a73446447e admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:01:25.709471 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde61910-5a0b-47a7-a971-e852fcf98252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:01:25.710354 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde61910-5a0b-47a7-a971-e852fcf98252 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:25.710637 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde61910-5a0b-47a7-a971-e852fcf98252 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:25.710920 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde61910-5a0b-47a7-a971-e852fcf98252 admin admin] PATH_INFO: `/v3/users/fdac43006996421799abf5bd401d0fc0` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:25.712457 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde61910-5a0b-47a7-a971-e852fcf98252 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdac43006996421799abf5bd401d0fc0)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:01:25.714459 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde61910-5a0b-47a7-a971-e852fcf98252 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:01:25.762150 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-cde61910-5a0b-47a7-a971-e852fcf98252 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdac43006996421799abf5bd401d0fc0'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:01:25.769707 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-cde61910-5a0b-47a7-a971-e852fcf98252 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdac43006996421799abf5bd401d0fc0'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:01:25.776799 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-cde61910-5a0b-47a7-a971-e852fcf98252 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdac43006996421799abf5bd401d0fc0'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:01:25.777467 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-cde61910-5a0b-47a7-a971-e852fcf98252 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdac43006996421799abf5bd401d0fc0'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:01:25.778083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-cde61910-5a0b-47a7-a971-e852fcf98252 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdac43006996421799abf5bd401d0fc0'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:01:25.796303 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1151/2286] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:01:25 2022] DELETE /identity/v3/users/fdac43006996421799abf5bd401d0fc0 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:01:25.978141 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a139d76-1ac2-4a0a-90d8-01644b33883e service placement] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:01:25.989714 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a139d76-1ac2-4a0a-90d8-01644b33883e service placement] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:01:26.056679 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1865c804-fb31-4ffd-837e-3bb8f85990e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:01:26.057563 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1865c804-fb31-4ffd-837e-3bb8f85990e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:26.057987 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1865c804-fb31-4ffd-837e-3bb8f85990e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:26.058351 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1865c804-fb31-4ffd-837e-3bb8f85990e0 admin admin] PATH_INFO: `/v3/projects/0124bfd61ad14bd395ecd3c54849f2fe` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:26.059783 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1865c804-fb31-4ffd-837e-3bb8f85990e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0124bfd61ad14bd395ecd3c54849f2fe)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:01:26.061786 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1865c804-fb31-4ffd-837e-3bb8f85990e0 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:01:26.108787 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-1865c804-fb31-4ffd-837e-3bb8f85990e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0124bfd61ad14bd395ecd3c54849f2fe', 'request_id': 'req-1865c804-fb31-4ffd-837e-3bb8f85990e0'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:01:26.114304 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-1865c804-fb31-4ffd-837e-3bb8f85990e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0124bfd61ad14bd395ecd3c54849f2fe', 'request_id': 'req-1865c804-fb31-4ffd-837e-3bb8f85990e0'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:01:26.122436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-1865c804-fb31-4ffd-837e-3bb8f85990e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0124bfd61ad14bd395ecd3c54849f2fe', 'request_id': 'req-1865c804-fb31-4ffd-837e-3bb8f85990e0'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:01:26.135341 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-1865c804-fb31-4ffd-837e-3bb8f85990e0 admin admin] The token cache is being invalidate because project 0124bfd61ad14bd395ecd3c54849f2fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:01:26.135646 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-1865c804-fb31-4ffd-837e-3bb8f85990e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:01:26.137584 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-1865c804-fb31-4ffd-837e-3bb8f85990e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:01:26.140036 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1136/2287] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:01:25 2022] DELETE /identity/v3/projects/0124bfd61ad14bd395ecd3c54849f2fe => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:01:26.145358 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c9ed6-3a81-4e28-b371-e8a72192f60c None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:26.145651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c9ed6-3a81-4e28-b371-e8a72192f60c None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:26.146104 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c9ed6-3a81-4e28-b371-e8a72192f60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:26.167775 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-4c7c9ed6-3a81-4e28-b371-e8a72192f60c None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:01:26.169144 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-4c7c9ed6-3a81-4e28-b371-e8a72192f60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:01:26.249844 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1152/2288] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:01:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:01:26.251331 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a1458-d0ad-4963-9acc-a358cb8e8b1c None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:26.251744 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a1458-d0ad-4963-9acc-a358cb8e8b1c None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:26.252087 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a1458-d0ad-4963-9acc-a358cb8e8b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:26.278835 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-d16a1458-d0ad-4963-9acc-a358cb8e8b1c None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:01:26.280312 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d16a1458-d0ad-4963-9acc-a358cb8e8b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:01:26.290939 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1137/2289] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:01:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 15:01:26.297631 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7c9ed6-3a81-4e28-b371-e8a72192f60c None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:01:26.310779 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7c9ed6-3a81-4e28-b371-e8a72192f60c None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:01:26.315549 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa494ca-8315-4e0d-a5ca-84bbd05cd4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:01:26.316224 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa494ca-8315-4e0d-a5ca-84bbd05cd4eb admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:26.317295 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa494ca-8315-4e0d-a5ca-84bbd05cd4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:26.317295 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa494ca-8315-4e0d-a5ca-84bbd05cd4eb admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:26.317726 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa494ca-8315-4e0d-a5ca-84bbd05cd4eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 15:01:26.318039 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa494ca-8315-4e0d-a5ca-84bbd05cd4eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:01:26.319637 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa494ca-8315-4e0d-a5ca-84bbd05cd4eb admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:01:26.335806 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1153/2290] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 15:01:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 15:01:26.337030 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a1458-d0ad-4963-9acc-a358cb8e8b1c None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:01:26.347492 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a1458-d0ad-4963-9acc-a358cb8e8b1c None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:01:26.352287 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3255fbea-2942-4673-874e-df7738864fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:01:26.353065 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3255fbea-2942-4673-874e-df7738864fe5 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:26.353282 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3255fbea-2942-4673-874e-df7738864fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:26.353557 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3255fbea-2942-4673-874e-df7738864fe5 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:26.354285 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3255fbea-2942-4673-874e-df7738864fe5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:01:26.356820 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3255fbea-2942-4673-874e-df7738864fe5 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:01:26.386336 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1138/2291] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:01:26 2022] POST /identity/v3/projects => generated 367 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:01:26.390828 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa494ca-8315-4e0d-a5ca-84bbd05cd4eb admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:01:26.401202 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa494ca-8315-4e0d-a5ca-84bbd05cd4eb admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:01:26.463709 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731da4c-6932-408d-a66e-3106e61ddf75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:01:26.465185 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731da4c-6932-408d-a66e-3106e61ddf75 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:26.465535 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731da4c-6932-408d-a66e-3106e61ddf75 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:26.465823 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731da4c-6932-408d-a66e-3106e61ddf75 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:26.466488 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8731da4c-6932-408d-a66e-3106e61ddf75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:01:26.468635 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8731da4c-6932-408d-a66e-3106e61ddf75 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:01:26.508553 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1154/2292] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:01:26 2022] POST /identity/v3/users => generated 284 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:01:26.509910 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3255fbea-2942-4673-874e-df7738864fe5 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:01:26.521503 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3255fbea-2942-4673-874e-df7738864fe5 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:01:26.526356 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8d7c6c-9425-448e-a932-50965dc6ecad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:01:26.527832 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d7c6c-9425-448e-a932-50965dc6ecad admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:26.528119 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d7c6c-9425-448e-a932-50965dc6ecad admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:26.528397 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d7c6c-9425-448e-a932-50965dc6ecad admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:26.529186 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8d7c6c-9425-448e-a932-50965dc6ecad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:01:26.531063 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8d7c6c-9425-448e-a932-50965dc6ecad admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:01:26.543325 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1139/2293] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:01:26.547456 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731da4c-6932-408d-a66e-3106e61ddf75 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:01:26.557442 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731da4c-6932-408d-a66e-3106e61ddf75 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:01:26.562470 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af57a37-b0ff-4f2b-abb5-46ea8190f8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:01:26.563053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af57a37-b0ff-4f2b-abb5-46ea8190f8f9 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:26.563342 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af57a37-b0ff-4f2b-abb5-46ea8190f8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:26.563625 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af57a37-b0ff-4f2b-abb5-46ea8190f8f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:26.564342 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af57a37-b0ff-4f2b-abb5-46ea8190f8f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:01:26.565982 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af57a37-b0ff-4f2b-abb5-46ea8190f8f9 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:01:26.576347 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1155/2294] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:01:26.580570 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8d7c6c-9425-448e-a932-50965dc6ecad admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:01:26.591376 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8d7c6c-9425-448e-a932-50965dc6ecad admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:01:26.596023 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf6d8ef-842d-4462-bf75-4058cd903992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:01:26.596933 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf6d8ef-842d-4462-bf75-4058cd903992 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:26.597221 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf6d8ef-842d-4462-bf75-4058cd903992 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:26.597495 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf6d8ef-842d-4462-bf75-4058cd903992 admin admin] PATH_INFO: `/v3/projects/a9e6da1cde894ad3a8da025915cea72c/users/39cbbc461dac4c55a165adb13fbb2965/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:26.613087 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf6d8ef-842d-4462-bf75-4058cd903992 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9e6da1cde894ad3a8da025915cea72c, user_id=39cbbc461dac4c55a165adb13fbb2965, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:01:26.615753 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf6d8ef-842d-4462-bf75-4058cd903992 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:01:26.628634 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1140/2295] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:01:26 2022] PUT /identity/v3/projects/a9e6da1cde894ad3a8da025915cea72c/users/39cbbc461dac4c55a165adb13fbb2965/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:01:26.640389 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e359bca-a6a0-46f9-a999-44ebcf7ccce1 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:26.640940 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e359bca-a6a0-46f9-a999-44ebcf7ccce1 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:26.641037 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e359bca-a6a0-46f9-a999-44ebcf7ccce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:26.668206 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-5e359bca-a6a0-46f9-a999-44ebcf7ccce1 None None] MFA Rules not processed for user `39cbbc461dac4c55a165adb13fbb2965`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:01:26.669447 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-5e359bca-a6a0-46f9-a999-44ebcf7ccce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:01:26.734389 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1156/2296] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:01:26 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:01:26.746223 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf6d8ef-842d-4462-bf75-4058cd903992 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:01:26.747966 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-bcf6d8ef-842d-4462-bf75-4058cd903992 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:01:26.849162 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf6d8ef-842d-4462-bf75-4058cd903992 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:01:26.853218 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed60596-7872-49b1-bbbb-b81f64e7d517 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9be93f9ff7e46babc8ee87b50b9c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:01:26.853973 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed60596-7872-49b1-bbbb-b81f64e7d517 service glance] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:26.854354 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed60596-7872-49b1-bbbb-b81f64e7d517 service glance] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:26.854829 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed60596-7872-49b1-bbbb-b81f64e7d517 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:26.864775 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed60596-7872-49b1-bbbb-b81f64e7d517 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:01:26.866615 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed60596-7872-49b1-bbbb-b81f64e7d517 service glance] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:01:26.885049 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1141/2297] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:01:26 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 15:01:27.384143 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e359bca-a6a0-46f9-a999-44ebcf7ccce1 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:01:27.385640 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-5e359bca-a6a0-46f9-a999-44ebcf7ccce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:01:27.499910 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e359bca-a6a0-46f9-a999-44ebcf7ccce1 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:01:27.506038 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b1679-6da0-4de1-b8d2-23e028e1d566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:01:27.507520 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b1679-6da0-4de1-b8d2-23e028e1d566 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:27.507850 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b1679-6da0-4de1-b8d2-23e028e1d566 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:27.514536 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b1679-6da0-4de1-b8d2-23e028e1d566 admin admin] PATH_INFO: `/v3/users/ce54cb83c2214551b0d472a960194ffb` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:27.516462 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8b1679-6da0-4de1-b8d2-23e028e1d566 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce54cb83c2214551b0d472a960194ffb)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:01:27.518881 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8b1679-6da0-4de1-b8d2-23e028e1d566 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:01:27.583353 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-6e8b1679-6da0-4de1-b8d2-23e028e1d566 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce54cb83c2214551b0d472a960194ffb'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:01:27.592116 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-6e8b1679-6da0-4de1-b8d2-23e028e1d566 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce54cb83c2214551b0d472a960194ffb'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:01:27.600897 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-6e8b1679-6da0-4de1-b8d2-23e028e1d566 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce54cb83c2214551b0d472a960194ffb'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:01:27.601952 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-6e8b1679-6da0-4de1-b8d2-23e028e1d566 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce54cb83c2214551b0d472a960194ffb'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:01:27.602773 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-6e8b1679-6da0-4de1-b8d2-23e028e1d566 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce54cb83c2214551b0d472a960194ffb'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:01:27.634226 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1157/2298] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:01:27 2022] DELETE /identity/v3/users/ce54cb83c2214551b0d472a960194ffb => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:01:27.639763 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f3ce72-0f4e-4e11-900a-760fbb36362c None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:27.640316 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f3ce72-0f4e-4e11-900a-760fbb36362c None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:27.640449 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f3ce72-0f4e-4e11-900a-760fbb36362c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:27.666385 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-55f3ce72-0f4e-4e11-900a-760fbb36362c None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:01:27.669173 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-55f3ce72-0f4e-4e11-900a-760fbb36362c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:01:27.744576 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1142/2299] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:01:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:01:27.751049 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b1679-6da0-4de1-b8d2-23e028e1d566 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:01:27.753198 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-6e8b1679-6da0-4de1-b8d2-23e028e1d566 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:01:27.827644 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b1679-6da0-4de1-b8d2-23e028e1d566 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:01:27.831401 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6512e8c2-a4ab-4471-9809-bce3ebf8c1b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:01:27.832132 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6512e8c2-a4ab-4471-9809-bce3ebf8c1b5 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:27.832490 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6512e8c2-a4ab-4471-9809-bce3ebf8c1b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:27.832875 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6512e8c2-a4ab-4471-9809-bce3ebf8c1b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:27.844520 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6512e8c2-a4ab-4471-9809-bce3ebf8c1b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:01:27.846072 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6512e8c2-a4ab-4471-9809-bce3ebf8c1b5 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:01:27.857347 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1158/2300] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:01:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:01:28.168733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f3ce72-0f4e-4e11-900a-760fbb36362c None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:01:28.170257 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-55f3ce72-0f4e-4e11-900a-760fbb36362c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:01:28.204434 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f3ce72-0f4e-4e11-900a-760fbb36362c None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:01:28.221226 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c3a41-c614-417b-bcea-74f4fc2923ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:01:28.222280 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c3a41-c614-417b-bcea-74f4fc2923ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:28.222686 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c3a41-c614-417b-bcea-74f4fc2923ba admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:28.223113 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c3a41-c614-417b-bcea-74f4fc2923ba admin admin] PATH_INFO: `/v3/projects/8199821451d1492aaf260e2bc556f580` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:28.224384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099c3a41-c614-417b-bcea-74f4fc2923ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=8199821451d1492aaf260e2bc556f580)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:01:28.226374 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099c3a41-c614-417b-bcea-74f4fc2923ba admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:01:28.285351 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-099c3a41-c614-417b-bcea-74f4fc2923ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8199821451d1492aaf260e2bc556f580', 'request_id': 'req-099c3a41-c614-417b-bcea-74f4fc2923ba'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:01:28.290989 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-099c3a41-c614-417b-bcea-74f4fc2923ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8199821451d1492aaf260e2bc556f580', 'request_id': 'req-099c3a41-c614-417b-bcea-74f4fc2923ba'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:01:28.306642 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-099c3a41-c614-417b-bcea-74f4fc2923ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8199821451d1492aaf260e2bc556f580', 'request_id': 'req-099c3a41-c614-417b-bcea-74f4fc2923ba'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:01:28.322495 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-099c3a41-c614-417b-bcea-74f4fc2923ba admin admin] The token cache is being invalidate because project 8199821451d1492aaf260e2bc556f580 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:01:28.322869 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-099c3a41-c614-417b-bcea-74f4fc2923ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:01:28.324445 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-099c3a41-c614-417b-bcea-74f4fc2923ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:01:28.326572 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1143/2301] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:01:28 2022] DELETE /identity/v3/projects/8199821451d1492aaf260e2bc556f580 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:01:28.331629 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99345b00-649f-4a34-99d3-1ce9dfe8bc32 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:28.331924 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99345b00-649f-4a34-99d3-1ce9dfe8bc32 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:28.332202 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99345b00-649f-4a34-99d3-1ce9dfe8bc32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:28.352010 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-99345b00-649f-4a34-99d3-1ce9dfe8bc32 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:01:28.353326 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-99345b00-649f-4a34-99d3-1ce9dfe8bc32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:01:28.429551 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1159/2302] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:01:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:01:28.439947 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd1a00-8bc2-464a-b231-21b5e7028042 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:28.440235 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd1a00-8bc2-464a-b231-21b5e7028042 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:28.440947 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd1a00-8bc2-464a-b231-21b5e7028042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:28.463789 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-36cd1a00-8bc2-464a-b231-21b5e7028042 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:01:28.465692 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-36cd1a00-8bc2-464a-b231-21b5e7028042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:01:28.476458 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1144/2303] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:01:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 15:01:28.481303 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99345b00-649f-4a34-99d3-1ce9dfe8bc32 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:01:28.492305 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99345b00-649f-4a34-99d3-1ce9dfe8bc32 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:01:28.498817 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c603b-953a-4a42-b8bd-5b792b20d8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:01:28.500595 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c603b-953a-4a42-b8bd-5b792b20d8e2 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:28.501237 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c603b-953a-4a42-b8bd-5b792b20d8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:28.502212 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c603b-953a-4a42-b8bd-5b792b20d8e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:28.502969 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795c603b-953a-4a42-b8bd-5b792b20d8e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 15:01:28.503279 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795c603b-953a-4a42-b8bd-5b792b20d8e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:01:28.506501 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795c603b-953a-4a42-b8bd-5b792b20d8e2 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:01:28.529093 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1160/2304] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 15:01:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 15:01:28.533952 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cd1a00-8bc2-464a-b231-21b5e7028042 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:01:28.544943 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cd1a00-8bc2-464a-b231-21b5e7028042 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:01:28.550018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c76dfb7-cf51-4fea-ae10-e90a62ea9af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:01:28.550692 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76dfb7-cf51-4fea-ae10-e90a62ea9af5 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:28.550975 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76dfb7-cf51-4fea-ae10-e90a62ea9af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:28.551252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76dfb7-cf51-4fea-ae10-e90a62ea9af5 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:28.551943 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c76dfb7-cf51-4fea-ae10-e90a62ea9af5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:01:28.554038 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c76dfb7-cf51-4fea-ae10-e90a62ea9af5 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:01:28.592952 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1145/2305] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:01:28 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:01:28.593255 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c603b-953a-4a42-b8bd-5b792b20d8e2 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:01:28.604204 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c603b-953a-4a42-b8bd-5b792b20d8e2 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:01:28.693971 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3690a483-f7ae-4449-885b-1c9d30601edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:01:28.694562 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690a483-f7ae-4449-885b-1c9d30601edb admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:28.694855 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690a483-f7ae-4449-885b-1c9d30601edb admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:28.695132 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690a483-f7ae-4449-885b-1c9d30601edb admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:28.695826 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3690a483-f7ae-4449-885b-1c9d30601edb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:01:28.697838 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3690a483-f7ae-4449-885b-1c9d30601edb admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:01:28.729076 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c76dfb7-cf51-4fea-ae10-e90a62ea9af5 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:01:28.736764 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1161/2306] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:01:28 2022] POST /identity/v3/users => generated 293 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 15:01:28.744710 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c76dfb7-cf51-4fea-ae10-e90a62ea9af5 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:01:28.751149 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd931a-ec7a-40a9-8371-e901c56c7baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:01:28.751956 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd931a-ec7a-40a9-8371-e901c56c7baa admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:28.752316 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd931a-ec7a-40a9-8371-e901c56c7baa admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:28.752821 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd931a-ec7a-40a9-8371-e901c56c7baa admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:28.753630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cd931a-ec7a-40a9-8371-e901c56c7baa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:01:28.755323 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cd931a-ec7a-40a9-8371-e901c56c7baa admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:01:28.768042 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1146/2307] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:01:28 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:01:28.772489 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3690a483-f7ae-4449-885b-1c9d30601edb admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:01:28.784756 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3690a483-f7ae-4449-885b-1c9d30601edb admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:01:28.789567 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e7e0c-2c33-4b6b-9c9a-1f8b6090dd35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:01:28.790478 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e7e0c-2c33-4b6b-9c9a-1f8b6090dd35 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:28.790767 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e7e0c-2c33-4b6b-9c9a-1f8b6090dd35 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:28.791305 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e7e0c-2c33-4b6b-9c9a-1f8b6090dd35 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:28.792841 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0e7e0c-2c33-4b6b-9c9a-1f8b6090dd35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:01:28.794625 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0e7e0c-2c33-4b6b-9c9a-1f8b6090dd35 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:01:28.805847 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1162/2308] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:01:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 06 15:01:28.810329 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd931a-ec7a-40a9-8371-e901c56c7baa admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:01:28.822274 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd931a-ec7a-40a9-8371-e901c56c7baa admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:01:28.829978 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382b81b-213d-4209-8329-e0240bec0a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:01:28.831008 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382b81b-213d-4209-8329-e0240bec0a71 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:28.831524 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382b81b-213d-4209-8329-e0240bec0a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:28.831871 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382b81b-213d-4209-8329-e0240bec0a71 admin admin] PATH_INFO: `/v3/projects/4be2354633354d249a0598f589a987b8/users/22199a431b8a42789bbc0f5e30b816ec/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:28.870334 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7382b81b-213d-4209-8329-e0240bec0a71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4be2354633354d249a0598f589a987b8, user_id=22199a431b8a42789bbc0f5e30b816ec, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:01:28.876237 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7382b81b-213d-4209-8329-e0240bec0a71 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:01:28.896183 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1147/2309] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:01:28 2022] PUT /identity/v3/projects/4be2354633354d249a0598f589a987b8/users/22199a431b8a42789bbc0f5e30b816ec/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 15:01:28.902279 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ea864-60a2-4042-858a-81245198d93b None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:28.902854 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ea864-60a2-4042-858a-81245198d93b None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:28.903320 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ea864-60a2-4042-858a-81245198d93b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:28.931270 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-7f2ea864-60a2-4042-858a-81245198d93b None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:01:28.933207 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-7f2ea864-60a2-4042-858a-81245198d93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:01:29.017505 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1163/2310] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:01:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 15:01:29.039087 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382b81b-213d-4209-8329-e0240bec0a71 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:01:29.040888 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-7382b81b-213d-4209-8329-e0240bec0a71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:01:29.131555 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382b81b-213d-4209-8329-e0240bec0a71 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:01:29.141845 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59047aa-553e-4339-a2ed-06e5e8b5d939 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:01:29.142478 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59047aa-553e-4339-a2ed-06e5e8b5d939 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:29.142810 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59047aa-553e-4339-a2ed-06e5e8b5d939 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:29.143092 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59047aa-553e-4339-a2ed-06e5e8b5d939 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:29.160471 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59047aa-553e-4339-a2ed-06e5e8b5d939 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:01:29.162195 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59047aa-553e-4339-a2ed-06e5e8b5d939 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:01:29.176077 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1148/2311] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:01:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:01:34.750882 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c656c0-1b49-42c8-a5f7-53691b378e71 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:34.751308 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c656c0-1b49-42c8-a5f7-53691b378e71 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:34.751606 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c656c0-1b49-42c8-a5f7-53691b378e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:34.787603 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-50c656c0-1b49-42c8-a5f7-53691b378e71 None None] MFA Rules not processed for user `22199a431b8a42789bbc0f5e30b816ec`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:01:34.788860 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-50c656c0-1b49-42c8-a5f7-53691b378e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:01:34.870933 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1164/2312] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:01:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:01:34.875452 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59047aa-553e-4339-a2ed-06e5e8b5d939 service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:01:34.890625 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59047aa-553e-4339-a2ed-06e5e8b5d939 service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:01:34.895332 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3f12f0-3d11-4abb-8602-b68b12503e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:01:34.895987 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f12f0-3d11-4abb-8602-b68b12503e8d admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:34.896455 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f12f0-3d11-4abb-8602-b68b12503e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:34.896779 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f12f0-3d11-4abb-8602-b68b12503e8d admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:34.897490 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3f12f0-3d11-4abb-8602-b68b12503e8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:01:34.899374 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3f12f0-3d11-4abb-8602-b68b12503e8d admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:01:34.928807 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1149/2313] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:01:34 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:01:34.930074 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c656c0-1b49-42c8-a5f7-53691b378e71 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:01:34.941822 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c656c0-1b49-42c8-a5f7-53691b378e71 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:01:35.004042 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e44238-baf6-41f2-a1d1-313d98941a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:01:35.004794 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e44238-baf6-41f2-a1d1-313d98941a5b admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:35.005084 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e44238-baf6-41f2-a1d1-313d98941a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:35.005364 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e44238-baf6-41f2-a1d1-313d98941a5b admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:35.006051 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e44238-baf6-41f2-a1d1-313d98941a5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:01:35.007969 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e44238-baf6-41f2-a1d1-313d98941a5b admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:01:35.028202 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1165/2314] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:01:34 2022] POST /identity/v3/users => generated 299 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:01:35.032820 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3f12f0-3d11-4abb-8602-b68b12503e8d admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:01:35.048678 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3f12f0-3d11-4abb-8602-b68b12503e8d admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:01:35.053686 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223e6e66-1883-4492-8518-fd96b1956cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:01:35.054400 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e6e66-1883-4492-8518-fd96b1956cc6 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:35.054685 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e6e66-1883-4492-8518-fd96b1956cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:35.054963 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e6e66-1883-4492-8518-fd96b1956cc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:35.055691 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223e6e66-1883-4492-8518-fd96b1956cc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:01:35.057460 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223e6e66-1883-4492-8518-fd96b1956cc6 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:01:35.076724 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1150/2315] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:01:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:01:35.078282 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e44238-baf6-41f2-a1d1-313d98941a5b admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:01:35.088922 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e44238-baf6-41f2-a1d1-313d98941a5b admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:01:35.094398 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f346a55-525b-4e1c-945e-28aaa6f73404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:01:35.095577 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f346a55-525b-4e1c-945e-28aaa6f73404 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:35.095900 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f346a55-525b-4e1c-945e-28aaa6f73404 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:35.096185 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f346a55-525b-4e1c-945e-28aaa6f73404 admin admin] PATH_INFO: `/v3/projects/2e9600e4323648cd9c7794d67bce9fe3/users/667e72222b0b4a9eadfd358358070782/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:35.112801 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f346a55-525b-4e1c-945e-28aaa6f73404 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e9600e4323648cd9c7794d67bce9fe3, user_id=667e72222b0b4a9eadfd358358070782, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:01:35.115668 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f346a55-525b-4e1c-945e-28aaa6f73404 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:01:35.130935 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1166/2316] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:01:35 2022] PUT /identity/v3/projects/2e9600e4323648cd9c7794d67bce9fe3/users/667e72222b0b4a9eadfd358358070782/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 15:01:35.134576 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223e6e66-1883-4492-8518-fd96b1956cc6 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:01:35.147001 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223e6e66-1883-4492-8518-fd96b1956cc6 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:01:35.226595 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1627f0-e4d2-40cf-b855-8fbf1b41dc11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:01:35.227288 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1627f0-e4d2-40cf-b855-8fbf1b41dc11 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:35.227558 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1627f0-e4d2-40cf-b855-8fbf1b41dc11 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:35.227833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1627f0-e4d2-40cf-b855-8fbf1b41dc11 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:35.228552 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1627f0-e4d2-40cf-b855-8fbf1b41dc11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:01:35.230765 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1627f0-e4d2-40cf-b855-8fbf1b41dc11 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:01:35.242745 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1151/2317] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:01:35 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:01:35.248445 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f346a55-525b-4e1c-945e-28aaa6f73404 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:01:35.260569 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f346a55-525b-4e1c-945e-28aaa6f73404 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:01:35.267060 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51996d73-ef60-49b5-975e-41df49b6cc1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:01:35.268502 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51996d73-ef60-49b5-975e-41df49b6cc1c admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:35.268843 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51996d73-ef60-49b5-975e-41df49b6cc1c admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:35.269124 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51996d73-ef60-49b5-975e-41df49b6cc1c admin admin] PATH_INFO: `/v3/projects/2e9600e4323648cd9c7794d67bce9fe3/users/667e72222b0b4a9eadfd358358070782/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:35.271211 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51996d73-ef60-49b5-975e-41df49b6cc1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e9600e4323648cd9c7794d67bce9fe3, user_id=667e72222b0b4a9eadfd358358070782, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:01:35.276305 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51996d73-ef60-49b5-975e-41df49b6cc1c admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:01:35.295896 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1167/2318] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:01:35 2022] PUT /identity/v3/projects/2e9600e4323648cd9c7794d67bce9fe3/users/667e72222b0b4a9eadfd358358070782/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:01:36.760350 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ddea5-6cab-481e-9981-429b8421d06a None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:36.761844 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ddea5-6cab-481e-9981-429b8421d06a None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:36.762358 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ddea5-6cab-481e-9981-429b8421d06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:36.785800 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-c32ddea5-6cab-481e-9981-429b8421d06a None None] MFA Rules not processed for user `81c9e87c348c4a2c82e8327a2cadfc5d`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:01:36.787305 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-c32ddea5-6cab-481e-9981-429b8421d06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:01:36.856871 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1152/2319] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:01:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:01:41.446328 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4cb073-4d02-4dbd-8314-b3f1512ece43 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:41.446917 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4cb073-4d02-4dbd-8314-b3f1512ece43 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:41.447155 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4cb073-4d02-4dbd-8314-b3f1512ece43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:41.496400 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-da4cb073-4d02-4dbd-8314-b3f1512ece43 None None] MFA Rules not processed for user `667e72222b0b4a9eadfd358358070782`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:01:41.497926 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-da4cb073-4d02-4dbd-8314-b3f1512ece43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:01:41.610528 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1168/2320] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:01:41 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:01:43.487530 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32ddea5-6cab-481e-9981-429b8421d06a None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:01:43.489324 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-c32ddea5-6cab-481e-9981-429b8421d06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:01:43.578119 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32ddea5-6cab-481e-9981-429b8421d06a None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:01:43.582196 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9611cbca-7e28-421a-ae9e-530b4f876df8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9be93f9ff7e46babc8ee87b50b9c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:01:43.582946 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9611cbca-7e28-421a-ae9e-530b4f876df8 service glance] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:43.583337 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9611cbca-7e28-421a-ae9e-530b4f876df8 service glance] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:43.583651 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9611cbca-7e28-421a-ae9e-530b4f876df8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:43.658444 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9611cbca-7e28-421a-ae9e-530b4f876df8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:01:43.659805 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9611cbca-7e28-421a-ae9e-530b4f876df8 service glance] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:01:43.674628 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1153/2321] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:01:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:01:43.845403 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a2c17-5065-4ff7-b9d0-55553a9bc5c3 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:43.845868 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a2c17-5065-4ff7-b9d0-55553a9bc5c3 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:43.845950 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a2c17-5065-4ff7-b9d0-55553a9bc5c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:43.906610 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-cb1a2c17-5065-4ff7-b9d0-55553a9bc5c3 None None] MFA Rules not processed for user `a7dc1bc53c59423e934f7831ea022c73`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:01:43.908839 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-cb1a2c17-5065-4ff7-b9d0-55553a9bc5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:01:44.019079 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1169/2322] 10.222.0.23 () {58 vars in 1009 bytes} [Fri May 6 15:01:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:01:44.048710 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9611cbca-7e28-421a-ae9e-530b4f876df8 service glance] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:01:44.050306 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-9611cbca-7e28-421a-ae9e-530b4f876df8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:01:44.159307 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9611cbca-7e28-421a-ae9e-530b4f876df8 service glance] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:01:44.170704 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f528fbd8-77a8-46c3-b383-488dbd716d92 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf9eed62c8f1409282c2af6488426a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:01:44.171383 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528fbd8-77a8-46c3-b383-488dbd716d92 service swift] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:44.172107 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528fbd8-77a8-46c3-b383-488dbd716d92 service swift] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:44.172107 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528fbd8-77a8-46c3-b383-488dbd716d92 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:44.182825 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f528fbd8-77a8-46c3-b383-488dbd716d92 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:01:44.184464 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f528fbd8-77a8-46c3-b383-488dbd716d92 service swift] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:01:44.209440 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1154/2323] 10.222.0.23 () {60 vars in 1384 bytes} [Fri May 6 15:01:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 06 15:01:46.534846 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a2c17-5065-4ff7-b9d0-55553a9bc5c3 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:01:46.670369 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a2c17-5065-4ff7-b9d0-55553a9bc5c3 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:01:46.676540 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d403d-6213-403c-a3be-ad1bcd6c751b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:01:46.691835 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d403d-6213-403c-a3be-ad1bcd6c751b service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:46.692443 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d403d-6213-403c-a3be-ad1bcd6c751b service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:46.695516 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d403d-6213-403c-a3be-ad1bcd6c751b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:46.705285 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-200d403d-6213-403c-a3be-ad1bcd6c751b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:01:46.857120 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200d403d-6213-403c-a3be-ad1bcd6c751b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:01:46.858864 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200d403d-6213-403c-a3be-ad1bcd6c751b service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:01:46.875604 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1170/2324] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:01:46 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 15:01:50.419167 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d629f-f4cc-4d19-86ce-b4b153ba5804 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:50.419766 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d629f-f4cc-4d19-86ce-b4b153ba5804 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:50.420716 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d629f-f4cc-4d19-86ce-b4b153ba5804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:50.456397 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-999d629f-f4cc-4d19-86ce-b4b153ba5804 None None] MFA Rules not processed for user `a7dc1bc53c59423e934f7831ea022c73`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:01:50.459848 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-999d629f-f4cc-4d19-86ce-b4b153ba5804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:01:50.482498 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1155/2325] 10.222.0.23 () {58 vars in 1009 bytes} [Fri May 6 15:01:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:01:50.496882 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d403d-6213-403c-a3be-ad1bcd6c751b service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:01:50.511829 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d403d-6213-403c-a3be-ad1bcd6c751b service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:01:50.518410 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb45c38-333e-4f30-a1a2-a7fd12b6b27f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf9eed62c8f1409282c2af6488426a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:01:50.519440 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb45c38-333e-4f30-a1a2-a7fd12b6b27f service swift] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:50.519946 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb45c38-333e-4f30-a1a2-a7fd12b6b27f service swift] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:50.520430 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb45c38-333e-4f30-a1a2-a7fd12b6b27f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:50.550989 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb45c38-333e-4f30-a1a2-a7fd12b6b27f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:01:50.557691 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb45c38-333e-4f30-a1a2-a7fd12b6b27f service swift] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:01:50.573695 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1171/2326] 10.222.0.23 () {60 vars in 1384 bytes} [Fri May 6 15:01:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 06 15:01:58.972564 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb7d69-914c-4207-99c6-55af04a9df63 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:01:58.973156 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb7d69-914c-4207-99c6-55af04a9df63 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:01:58.973371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb7d69-914c-4207-99c6-55af04a9df63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:01:58.993542 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-c4cb7d69-914c-4207-99c6-55af04a9df63 None None] MFA Rules not processed for user `22199a431b8a42789bbc0f5e30b816ec`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:01:58.995868 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-c4cb7d69-914c-4207-99c6-55af04a9df63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:01:59.004440 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1156/2327] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:01:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:02:03.864969 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6fc8e3-f56a-40ef-9163-d11e5a874344 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:02:03.866560 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6fc8e3-f56a-40ef-9163-d11e5a874344 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:02:03.867026 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6fc8e3-f56a-40ef-9163-d11e5a874344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:02:03.886612 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-bd6fc8e3-f56a-40ef-9163-d11e5a874344 None None] MFA Rules not processed for user `a7dc1bc53c59423e934f7831ea022c73`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:02:03.888098 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-bd6fc8e3-f56a-40ef-9163-d11e5a874344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:02:03.896705 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1172/2328] 10.222.0.23 () {58 vars in 1009 bytes} [Fri May 6 15:02:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:02:03.910245 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb7d69-914c-4207-99c6-55af04a9df63 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:02:03.930219 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb7d69-914c-4207-99c6-55af04a9df63 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:02:03.937240 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b019b90a-df3f-4f95-8a88-afc538283ef5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf9eed62c8f1409282c2af6488426a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:02:03.938856 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019b90a-df3f-4f95-8a88-afc538283ef5 service swift] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:02:03.939713 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019b90a-df3f-4f95-8a88-afc538283ef5 service swift] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:02:03.940179 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019b90a-df3f-4f95-8a88-afc538283ef5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:02:03.952592 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b019b90a-df3f-4f95-8a88-afc538283ef5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:02:03.956440 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b019b90a-df3f-4f95-8a88-afc538283ef5 service swift] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:02:03.992481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1157/2329] 10.222.0.23 () {60 vars in 1384 bytes} [Fri May 6 15:02:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 06 15:02:28.077441 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1a388-b8af-4655-b13e-ede81d608167 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:02:28.079267 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1a388-b8af-4655-b13e-ede81d608167 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:02:28.079792 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1a388-b8af-4655-b13e-ede81d608167 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:02:28.081766 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1173/2330] 10.222.0.23 () {54 vars in 921 bytes} [Fri May 6 15:02:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 15:02:28.093847 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e0fa3-4849-4489-a373-62f59af5f790 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:02:28.095294 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e0fa3-4849-4489-a373-62f59af5f790 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:02:28.096293 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e0fa3-4849-4489-a373-62f59af5f790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:02:28.182749 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-906e0fa3-4849-4489-a373-62f59af5f790 None None] MFA Rules not processed for user `fc8cf09642ec4e10a80badcf8b585ef8`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:02:28.183976 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-906e0fa3-4849-4489-a373-62f59af5f790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:02:28.267564 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1158/2331] 10.222.0.23 () {58 vars in 1018 bytes} [Fri May 6 15:02:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:02:28.281189 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c1a388-b8af-4655-b13e-ede81d608167 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:02:28.292782 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c1a388-b8af-4655-b13e-ede81d608167 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:02:28.296301 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da0cfe-1b76-4e67-9b92-5fc47de4b48e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9be93f9ff7e46babc8ee87b50b9c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:02:28.297016 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da0cfe-1b76-4e67-9b92-5fc47de4b48e service glance] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:02:28.297297 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da0cfe-1b76-4e67-9b92-5fc47de4b48e service glance] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:02:28.297573 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da0cfe-1b76-4e67-9b92-5fc47de4b48e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:02:28.308089 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38da0cfe-1b76-4e67-9b92-5fc47de4b48e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:02:28.309554 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38da0cfe-1b76-4e67-9b92-5fc47de4b48e service glance] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:02:28.322262 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1174/2332] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:02:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:02:28.393758 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccbf27e-1eb3-4eef-a78d-55071aaa43f7 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:02:28.394085 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccbf27e-1eb3-4eef-a78d-55071aaa43f7 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:02:28.394370 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccbf27e-1eb3-4eef-a78d-55071aaa43f7 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:02:28.395881 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1159/2333] 10.222.0.23 () {54 vars in 921 bytes} [Fri May 6 15:02:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 15:02:28.401079 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37361472-5e7e-44de-adae-221189acc58e None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:02:28.401445 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37361472-5e7e-44de-adae-221189acc58e None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:02:28.401786 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37361472-5e7e-44de-adae-221189acc58e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:02:28.424038 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-37361472-5e7e-44de-adae-221189acc58e None None] MFA Rules not processed for user `4dcd1542d8f643c4996142a88b9b6abb`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:02:28.425310 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-37361472-5e7e-44de-adae-221189acc58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:02:28.432876 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1175/2334] 10.222.0.23 () {58 vars in 1018 bytes} [Fri May 6 15:02:28 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:02:28.444577 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccbf27e-1eb3-4eef-a78d-55071aaa43f7 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:02:28.456089 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccbf27e-1eb3-4eef-a78d-55071aaa43f7 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:02:28.459672 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83591be4-c745-45d0-982e-877387363c93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:02:28.460300 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83591be4-c745-45d0-982e-877387363c93 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:02:28.460581 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83591be4-c745-45d0-982e-877387363c93 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:02:28.460886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83591be4-c745-45d0-982e-877387363c93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:02:28.468017 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83591be4-c745-45d0-982e-877387363c93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:02:28.470124 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83591be4-c745-45d0-982e-877387363c93 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:02:28.477691 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1160/2335] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:02:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:02:28.560015 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194770a5-54be-4421-980c-08af4b6d85d6 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:02:28.560305 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194770a5-54be-4421-980c-08af4b6d85d6 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:02:28.560579 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194770a5-54be-4421-980c-08af4b6d85d6 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:02:28.561973 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1176/2336] 10.222.0.23 () {54 vars in 921 bytes} [Fri May 6 15:02:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 15:02:28.567601 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a338513-71f9-49a4-9414-34e8c5c55a78 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:02:28.567893 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a338513-71f9-49a4-9414-34e8c5c55a78 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:02:28.568174 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a338513-71f9-49a4-9414-34e8c5c55a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:02:28.586533 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-9a338513-71f9-49a4-9414-34e8c5c55a78 None None] MFA Rules not processed for user `fc8cf09642ec4e10a80badcf8b585ef8`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:02:28.587734 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-9a338513-71f9-49a4-9414-34e8c5c55a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:02:28.597815 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1161/2337] 10.222.0.23 () {58 vars in 1018 bytes} [Fri May 6 15:02:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 15:02:28.608306 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194770a5-54be-4421-980c-08af4b6d85d6 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:02:28.619337 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194770a5-54be-4421-980c-08af4b6d85d6 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:02:28.622926 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b9b63-af94-40e3-9590-eee64344842d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:02:28.623566 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b9b63-af94-40e3-9590-eee64344842d service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:02:28.623850 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b9b63-af94-40e3-9590-eee64344842d service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:02:28.624128 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b9b63-af94-40e3-9590-eee64344842d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:02:28.634754 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956b9b63-af94-40e3-9590-eee64344842d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:02:28.636241 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956b9b63-af94-40e3-9590-eee64344842d service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:02:28.649148 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1177/2338] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:02:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:02:28.788533 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a338513-71f9-49a4-9414-34e8c5c55a78 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:02:28.790065 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-9a338513-71f9-49a4-9414-34e8c5c55a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:02:28.854584 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a338513-71f9-49a4-9414-34e8c5c55a78 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:02:28.860504 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838890a-a8da-4de3-a8eb-319d46a25661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:02:28.861426 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838890a-a8da-4de3-a8eb-319d46a25661 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:02:28.861702 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838890a-a8da-4de3-a8eb-319d46a25661 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:02:28.861972 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838890a-a8da-4de3-a8eb-319d46a25661 admin admin] PATH_INFO: `/v3/users/39cbbc461dac4c55a165adb13fbb2965` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:02:28.863163 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3838890a-a8da-4de3-a8eb-319d46a25661 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39cbbc461dac4c55a165adb13fbb2965)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:02:28.865116 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3838890a-a8da-4de3-a8eb-319d46a25661 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:02:28.914029 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-3838890a-a8da-4de3-a8eb-319d46a25661 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39cbbc461dac4c55a165adb13fbb2965'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:02:28.914644 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-3838890a-a8da-4de3-a8eb-319d46a25661 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39cbbc461dac4c55a165adb13fbb2965'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:02:28.924851 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-3838890a-a8da-4de3-a8eb-319d46a25661 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39cbbc461dac4c55a165adb13fbb2965'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:02:28.931950 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-3838890a-a8da-4de3-a8eb-319d46a25661 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39cbbc461dac4c55a165adb13fbb2965'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:02:28.932631 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-3838890a-a8da-4de3-a8eb-319d46a25661 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39cbbc461dac4c55a165adb13fbb2965'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:02:28.949467 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1162/2339] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:02:28 2022] DELETE /identity/v3/users/39cbbc461dac4c55a165adb13fbb2965 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:02:28.954226 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1096b4-397e-428f-9c6d-56494d26c55a None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:02:28.954521 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1096b4-397e-428f-9c6d-56494d26c55a None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:02:28.954805 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1096b4-397e-428f-9c6d-56494d26c55a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:02:28.972757 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-5e1096b4-397e-428f-9c6d-56494d26c55a None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:02:28.973994 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-5e1096b4-397e-428f-9c6d-56494d26c55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:02:29.038249 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1178/2340] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:02:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:02:29.048733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838890a-a8da-4de3-a8eb-319d46a25661 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:02:29.050210 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-3838890a-a8da-4de3-a8eb-319d46a25661 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:02:29.113258 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838890a-a8da-4de3-a8eb-319d46a25661 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:02:29.116960 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47180e35-257a-4b9c-87d0-6d4faeca7fea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:02:29.117616 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47180e35-257a-4b9c-87d0-6d4faeca7fea service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:02:29.117925 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47180e35-257a-4b9c-87d0-6d4faeca7fea service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:02:29.118340 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47180e35-257a-4b9c-87d0-6d4faeca7fea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:02:29.128721 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47180e35-257a-4b9c-87d0-6d4faeca7fea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:02:29.130185 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47180e35-257a-4b9c-87d0-6d4faeca7fea service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:02:29.141540 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1163/2341] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:02:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:02:29.441202 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1096b4-397e-428f-9c6d-56494d26c55a None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:02:29.442707 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-5e1096b4-397e-428f-9c6d-56494d26c55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:02:29.458143 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1096b4-397e-428f-9c6d-56494d26c55a None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:02:29.462904 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2792edf-79b4-46f3-afa2-a9bb21f9b352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:02:29.463971 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2792edf-79b4-46f3-afa2-a9bb21f9b352 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:02:29.464253 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2792edf-79b4-46f3-afa2-a9bb21f9b352 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:02:29.464530 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2792edf-79b4-46f3-afa2-a9bb21f9b352 admin admin] PATH_INFO: `/v3/projects/a9e6da1cde894ad3a8da025915cea72c` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:02:29.465852 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2792edf-79b4-46f3-afa2-a9bb21f9b352 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9e6da1cde894ad3a8da025915cea72c)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:02:29.467732 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2792edf-79b4-46f3-afa2-a9bb21f9b352 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:02:29.519571 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b2792edf-79b4-46f3-afa2-a9bb21f9b352 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9e6da1cde894ad3a8da025915cea72c', 'request_id': 'req-b2792edf-79b4-46f3-afa2-a9bb21f9b352'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:02:29.526020 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b2792edf-79b4-46f3-afa2-a9bb21f9b352 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9e6da1cde894ad3a8da025915cea72c', 'request_id': 'req-b2792edf-79b4-46f3-afa2-a9bb21f9b352'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:02:29.534657 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b2792edf-79b4-46f3-afa2-a9bb21f9b352 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9e6da1cde894ad3a8da025915cea72c', 'request_id': 'req-b2792edf-79b4-46f3-afa2-a9bb21f9b352'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:02:29.547404 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b2792edf-79b4-46f3-afa2-a9bb21f9b352 admin admin] The token cache is being invalidate because project a9e6da1cde894ad3a8da025915cea72c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:02:29.547691 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b2792edf-79b4-46f3-afa2-a9bb21f9b352 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:02:29.548358 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b2792edf-79b4-46f3-afa2-a9bb21f9b352 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:02:29.549938 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1179/2342] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:02:29 2022] DELETE /identity/v3/projects/a9e6da1cde894ad3a8da025915cea72c => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:02:29.554985 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae39648-6ae1-48b3-b3aa-286d081b9037 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:02:29.555280 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae39648-6ae1-48b3-b3aa-286d081b9037 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:02:29.555560 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae39648-6ae1-48b3-b3aa-286d081b9037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:02:29.574210 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-cae39648-6ae1-48b3-b3aa-286d081b9037 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:02:29.575545 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-cae39648-6ae1-48b3-b3aa-286d081b9037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:02:29.635500 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1164/2343] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:02:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:02:29.646869 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3ee1a4-a91a-4390-bf0d-b12ecebc66ed None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:02:29.647733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3ee1a4-a91a-4390-bf0d-b12ecebc66ed None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:02:29.647733 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3ee1a4-a91a-4390-bf0d-b12ecebc66ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:02:29.667554 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-3e3ee1a4-a91a-4390-bf0d-b12ecebc66ed None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:02:29.668858 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-3e3ee1a4-a91a-4390-bf0d-b12ecebc66ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:02:29.677977 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1180/2344] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:02:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:02:29.682817 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae39648-6ae1-48b3-b3aa-286d081b9037 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:02:29.693308 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae39648-6ae1-48b3-b3aa-286d081b9037 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:02:29.697918 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a8773-05a3-4ac6-9a73-a027026a7cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:02:29.698598 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a8773-05a3-4ac6-9a73-a027026a7cc1 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:02:29.698879 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a8773-05a3-4ac6-9a73-a027026a7cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:02:29.699203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a8773-05a3-4ac6-9a73-a027026a7cc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:02:29.699936 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9a8773-05a3-4ac6-9a73-a027026a7cc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 15:02:29.700304 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9a8773-05a3-4ac6-9a73-a027026a7cc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:02:29.703447 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9a8773-05a3-4ac6-9a73-a027026a7cc1 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:02:29.719603 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1165/2345] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 15:02:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 15:02:29.723815 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3ee1a4-a91a-4390-bf0d-b12ecebc66ed None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:02:29.736243 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3ee1a4-a91a-4390-bf0d-b12ecebc66ed None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:02:29.742848 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e6059a-1da0-4d72-98d2-02f058fc576e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:02:29.743550 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6059a-1da0-4d72-98d2-02f058fc576e admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:02:29.744034 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6059a-1da0-4d72-98d2-02f058fc576e admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:02:29.744381 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6059a-1da0-4d72-98d2-02f058fc576e admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:02:29.745645 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e6059a-1da0-4d72-98d2-02f058fc576e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:02:29.748429 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e6059a-1da0-4d72-98d2-02f058fc576e admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:02:29.800691 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1181/2346] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:02:29 2022] POST /identity/v3/projects => generated 373 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 15:02:29.804660 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a8773-05a3-4ac6-9a73-a027026a7cc1 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:02:29.814821 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a8773-05a3-4ac6-9a73-a027026a7cc1 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:02:29.868924 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06182f50-1f71-41d9-95b2-9d81e57f4a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:02:29.869596 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06182f50-1f71-41d9-95b2-9d81e57f4a2a admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:02:29.869894 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06182f50-1f71-41d9-95b2-9d81e57f4a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:02:29.870149 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06182f50-1f71-41d9-95b2-9d81e57f4a2a admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:02:29.870842 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06182f50-1f71-41d9-95b2-9d81e57f4a2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:02:29.872784 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06182f50-1f71-41d9-95b2-9d81e57f4a2a admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:02:29.896855 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1166/2347] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:02:29 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 15:02:29.901259 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e6059a-1da0-4d72-98d2-02f058fc576e admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:02:29.911779 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e6059a-1da0-4d72-98d2-02f058fc576e admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:02:29.916362 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e4f60-8359-417b-b4d0-a15c2d227b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:02:29.917052 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e4f60-8359-417b-b4d0-a15c2d227b5d admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:02:29.917337 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e4f60-8359-417b-b4d0-a15c2d227b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:02:29.917610 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e4f60-8359-417b-b4d0-a15c2d227b5d admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:02:29.918323 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917e4f60-8359-417b-b4d0-a15c2d227b5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:02:29.919902 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917e4f60-8359-417b-b4d0-a15c2d227b5d admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:02:29.930857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1182/2348] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:02:29.934864 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06182f50-1f71-41d9-95b2-9d81e57f4a2a admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:02:29.945546 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06182f50-1f71-41d9-95b2-9d81e57f4a2a admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:02:29.950140 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23437a1b-2baf-47b5-b170-e2d5efd8d8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:02:29.950790 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23437a1b-2baf-47b5-b170-e2d5efd8d8b9 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:02:29.951075 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23437a1b-2baf-47b5-b170-e2d5efd8d8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:02:29.951356 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23437a1b-2baf-47b5-b170-e2d5efd8d8b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:02:29.952069 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23437a1b-2baf-47b5-b170-e2d5efd8d8b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:02:29.953773 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23437a1b-2baf-47b5-b170-e2d5efd8d8b9 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:02:29.963662 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1167/2349] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:02:29.967690 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e4f60-8359-417b-b4d0-a15c2d227b5d admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:02:29.977606 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e4f60-8359-417b-b4d0-a15c2d227b5d admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:02:29.982264 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b03e59-ab9d-498a-9d3a-5e5d3a0d10c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:02:29.983137 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b03e59-ab9d-498a-9d3a-5e5d3a0d10c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:02:29.983420 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b03e59-ab9d-498a-9d3a-5e5d3a0d10c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:02:29.983697 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b03e59-ab9d-498a-9d3a-5e5d3a0d10c3 admin admin] PATH_INFO: `/v3/projects/9c9ebfd7aafd4a488325abebd305ffb3/users/989a604bec9e4089a54093ef3b9bf5ea/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:02:29.998479 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b03e59-ab9d-498a-9d3a-5e5d3a0d10c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c9ebfd7aafd4a488325abebd305ffb3, user_id=989a604bec9e4089a54093ef3b9bf5ea, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:02:30.001803 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b03e59-ab9d-498a-9d3a-5e5d3a0d10c3 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:02:30.014391 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1183/2350] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:02:29 2022] PUT /identity/v3/projects/9c9ebfd7aafd4a488325abebd305ffb3/users/989a604bec9e4089a54093ef3b9bf5ea/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:02:30.025741 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c4ccf-6433-493c-82c7-0150aa7a39ef None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:02:30.026038 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c4ccf-6433-493c-82c7-0150aa7a39ef None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:02:30.026329 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c4ccf-6433-493c-82c7-0150aa7a39ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:02:30.053584 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-968c4ccf-6433-493c-82c7-0150aa7a39ef None None] MFA Rules not processed for user `989a604bec9e4089a54093ef3b9bf5ea`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:02:30.054798 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-968c4ccf-6433-493c-82c7-0150aa7a39ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:02:30.122996 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1168/2351] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:02:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:02:30.135849 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b03e59-ab9d-498a-9d3a-5e5d3a0d10c3 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:02:30.137660 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-21b03e59-ab9d-498a-9d3a-5e5d3a0d10c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:02:30.237236 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b03e59-ab9d-498a-9d3a-5e5d3a0d10c3 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:02:30.241576 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b23566-9943-4d31-8927-1ae986baea1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:02:30.242639 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b23566-9943-4d31-8927-1ae986baea1a service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:02:30.242880 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b23566-9943-4d31-8927-1ae986baea1a service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:02:30.243026 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b23566-9943-4d31-8927-1ae986baea1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:02:30.251950 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b23566-9943-4d31-8927-1ae986baea1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:02:30.253836 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b23566-9943-4d31-8927-1ae986baea1a service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:02:30.263987 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1184/2352] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:02:30 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:02:31.622440 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d4183-05c3-4201-8224-4541122c5f6a None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:02:31.623032 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d4183-05c3-4201-8224-4541122c5f6a None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:02:31.623187 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d4183-05c3-4201-8224-4541122c5f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:02:31.642472 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-540d4183-05c3-4201-8224-4541122c5f6a None None] MFA Rules not processed for user `989a604bec9e4089a54093ef3b9bf5ea`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:02:31.643741 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-540d4183-05c3-4201-8224-4541122c5f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:02:31.651494 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1169/2353] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:02:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:02:48.150354 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a91c8-1b38-4a5c-964f-ae096e246a9f None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:02:48.150914 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a91c8-1b38-4a5c-964f-ae096e246a9f None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:02:48.151170 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a91c8-1b38-4a5c-964f-ae096e246a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:02:48.174277 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-878a91c8-1b38-4a5c-964f-ae096e246a9f None None] MFA Rules not processed for user `a7dc1bc53c59423e934f7831ea022c73`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:02:48.175652 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-878a91c8-1b38-4a5c-964f-ae096e246a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:02:48.255211 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1185/2354] 10.222.0.23 () {58 vars in 1009 bytes} [Fri May 6 15:02:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:02:48.280153 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540d4183-05c3-4201-8224-4541122c5f6a None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:02:48.282572 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-540d4183-05c3-4201-8224-4541122c5f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:02:48.355486 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540d4183-05c3-4201-8224-4541122c5f6a None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:02:48.360985 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f142e-010d-4da2-86ed-515f289d4516 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf9eed62c8f1409282c2af6488426a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:02:48.361660 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f142e-010d-4da2-86ed-515f289d4516 service swift] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:02:48.361944 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f142e-010d-4da2-86ed-515f289d4516 service swift] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:02:48.362222 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f142e-010d-4da2-86ed-515f289d4516 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:02:48.371565 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014f142e-010d-4da2-86ed-515f289d4516 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:02:48.373344 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014f142e-010d-4da2-86ed-515f289d4516 service swift] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:02:48.383138 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1170/2355] 10.222.0.23 () {60 vars in 1384 bytes} [Fri May 6 15:02:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 06 15:03:19.117812 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a91c8-1b38-4a5c-964f-ae096e246a9f None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:03:19.126078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-878a91c8-1b38-4a5c-964f-ae096e246a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:03:19.205575 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a91c8-1b38-4a5c-964f-ae096e246a9f None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:03:19.209346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada8b36e-d6be-4307-af3e-369181a527d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:03:19.210098 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8b36e-d6be-4307-af3e-369181a527d2 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:19.210451 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8b36e-d6be-4307-af3e-369181a527d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:19.210767 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8b36e-d6be-4307-af3e-369181a527d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:19.212891 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-ada8b36e-d6be-4307-af3e-369181a527d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:03:19.295121 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada8b36e-d6be-4307-af3e-369181a527d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:03:19.297490 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada8b36e-d6be-4307-af3e-369181a527d2 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:03:19.309135 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1186/2356] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:03:19 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:03:21.634268 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f142e-010d-4da2-86ed-515f289d4516 service swift] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:03:21.645979 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f142e-010d-4da2-86ed-515f289d4516 service swift] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:03:21.652204 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b244d0-9d3e-47e0-a3a1-7661794b62ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:03:21.653188 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b244d0-9d3e-47e0-a3a1-7661794b62ef service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:21.654125 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b244d0-9d3e-47e0-a3a1-7661794b62ef service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:21.654815 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b244d0-9d3e-47e0-a3a1-7661794b62ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:21.657802 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-42b244d0-9d3e-47e0-a3a1-7661794b62ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:03:21.678712 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada8b36e-d6be-4307-af3e-369181a527d2 service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:03:21.680888 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-ada8b36e-d6be-4307-af3e-369181a527d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:03:21.693613 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada8b36e-d6be-4307-af3e-369181a527d2 service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:03:21.697827 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975355ee-0274-4a10-bb2c-99a538a5de5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:03:21.698857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975355ee-0274-4a10-bb2c-99a538a5de5c service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:21.699903 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975355ee-0274-4a10-bb2c-99a538a5de5c service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:21.700377 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975355ee-0274-4a10-bb2c-99a538a5de5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:21.703485 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-975355ee-0274-4a10-bb2c-99a538a5de5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:03:21.770491 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b244d0-9d3e-47e0-a3a1-7661794b62ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:03:21.772318 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b244d0-9d3e-47e0-a3a1-7661794b62ef service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:03:21.786118 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975355ee-0274-4a10-bb2c-99a538a5de5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:03:21.788074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1171/2357] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:03:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:03:21.789147 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975355ee-0274-4a10-bb2c-99a538a5de5c service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:03:21.805562 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1187/2358] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:03:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 15:03:24.549888 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e598ea3-ca20-4ad5-acb1-f2c3241b21aa None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:24.550427 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e598ea3-ca20-4ad5-acb1-f2c3241b21aa None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:24.550534 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e598ea3-ca20-4ad5-acb1-f2c3241b21aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:24.569220 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-1e598ea3-ca20-4ad5-acb1-f2c3241b21aa None None] MFA Rules not processed for user `a7dc1bc53c59423e934f7831ea022c73`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:03:24.570457 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-1e598ea3-ca20-4ad5-acb1-f2c3241b21aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:03:24.578574 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1172/2359] 10.222.0.23 () {58 vars in 1009 bytes} [Fri May 6 15:03:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:03:24.591726 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975355ee-0274-4a10-bb2c-99a538a5de5c service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:03:24.603140 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975355ee-0274-4a10-bb2c-99a538a5de5c service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:03:24.609069 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f69c7-8ccd-490c-b1df-175ab7e82b9e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf9eed62c8f1409282c2af6488426a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:03:24.609785 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f69c7-8ccd-490c-b1df-175ab7e82b9e service swift] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:24.610145 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f69c7-8ccd-490c-b1df-175ab7e82b9e service swift] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:24.610483 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f69c7-8ccd-490c-b1df-175ab7e82b9e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:24.619842 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8f69c7-8ccd-490c-b1df-175ab7e82b9e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:03:24.621540 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8f69c7-8ccd-490c-b1df-175ab7e82b9e service swift] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:03:24.631263 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1188/2360] 10.222.0.23 () {60 vars in 1384 bytes} [Fri May 6 15:03:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 06 15:03:25.160790 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38692832-1ab5-42d3-8a71-f4a775a66e93 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:25.161087 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38692832-1ab5-42d3-8a71-f4a775a66e93 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:25.161378 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38692832-1ab5-42d3-8a71-f4a775a66e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:25.181538 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-38692832-1ab5-42d3-8a71-f4a775a66e93 None None] MFA Rules not processed for user `a7dc1bc53c59423e934f7831ea022c73`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:03:25.182821 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-38692832-1ab5-42d3-8a71-f4a775a66e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:03:25.190909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1173/2361] 10.222.0.23 () {58 vars in 967 bytes} [Fri May 6 15:03:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:03:25.204107 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f69c7-8ccd-490c-b1df-175ab7e82b9e service swift] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:03:25.214228 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f69c7-8ccd-490c-b1df-175ab7e82b9e service swift] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:03:25.219818 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e8f17-b985-46f2-af34-53a87fb1fe39 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf9eed62c8f1409282c2af6488426a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:03:25.220465 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e8f17-b985-46f2-af34-53a87fb1fe39 service swift] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:25.220853 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e8f17-b985-46f2-af34-53a87fb1fe39 service swift] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:25.221134 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e8f17-b985-46f2-af34-53a87fb1fe39 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:25.229986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2e8f17-b985-46f2-af34-53a87fb1fe39 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:03:25.231591 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2e8f17-b985-46f2-af34-53a87fb1fe39 service swift] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:03:25.241215 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1189/2362] 10.222.0.23 () {60 vars in 1384 bytes} [Fri May 6 15:03:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 06 15:03:25.473214 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa851c0-a8c4-4e59-8c77-835f6cdb4e6e None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:25.473503 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa851c0-a8c4-4e59-8c77-835f6cdb4e6e None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:25.473778 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa851c0-a8c4-4e59-8c77-835f6cdb4e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:25.491856 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-8fa851c0-a8c4-4e59-8c77-835f6cdb4e6e None None] MFA Rules not processed for user `a7dc1bc53c59423e934f7831ea022c73`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:03:25.493078 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-8fa851c0-a8c4-4e59-8c77-835f6cdb4e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:03:25.500952 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1174/2363] 10.222.0.23 () {58 vars in 1009 bytes} [Fri May 6 15:03:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:03:25.514010 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e8f17-b985-46f2-af34-53a87fb1fe39 service swift] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:03:25.525986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e8f17-b985-46f2-af34-53a87fb1fe39 service swift] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:03:25.535931 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bb6ac7-7cab-4b86-824f-3ce94fd28696 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf9eed62c8f1409282c2af6488426a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:03:25.536877 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb6ac7-7cab-4b86-824f-3ce94fd28696 service swift] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:25.537797 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb6ac7-7cab-4b86-824f-3ce94fd28696 service swift] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:25.538228 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb6ac7-7cab-4b86-824f-3ce94fd28696 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:25.553636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bb6ac7-7cab-4b86-824f-3ce94fd28696 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:03:25.557185 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bb6ac7-7cab-4b86-824f-3ce94fd28696 service swift] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:03:25.568127 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1190/2364] 10.222.0.23 () {60 vars in 1384 bytes} [Fri May 6 15:03:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 06 15:03:25.940854 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663f29e-05e5-4c88-820f-8af08352be6f None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:25.941323 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663f29e-05e5-4c88-820f-8af08352be6f None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:25.941444 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663f29e-05e5-4c88-820f-8af08352be6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:25.959860 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-3663f29e-05e5-4c88-820f-8af08352be6f None None] MFA Rules not processed for user `a7dc1bc53c59423e934f7831ea022c73`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:03:25.961111 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-3663f29e-05e5-4c88-820f-8af08352be6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:03:25.968402 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1175/2365] 10.222.0.23 () {58 vars in 967 bytes} [Fri May 6 15:03:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:03:25.980798 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bb6ac7-7cab-4b86-824f-3ce94fd28696 service swift] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:03:25.991083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bb6ac7-7cab-4b86-824f-3ce94fd28696 service swift] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:03:25.996069 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e342f5dd-7956-482b-8314-74a18d355e99 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf9eed62c8f1409282c2af6488426a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:03:25.996770 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342f5dd-7956-482b-8314-74a18d355e99 service swift] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:25.997078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342f5dd-7956-482b-8314-74a18d355e99 service swift] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:25.997473 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342f5dd-7956-482b-8314-74a18d355e99 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:26.006255 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e342f5dd-7956-482b-8314-74a18d355e99 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:03:26.007869 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e342f5dd-7956-482b-8314-74a18d355e99 service swift] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:03:26.016404 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1191/2366] 10.222.0.23 () {60 vars in 1384 bytes} [Fri May 6 15:03:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 06 15:03:29.930876 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae182a-b50f-4656-9641-0fa682e1d3c0 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:29.931405 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae182a-b50f-4656-9641-0fa682e1d3c0 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:29.931506 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae182a-b50f-4656-9641-0fa682e1d3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:29.956376 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-68ae182a-b50f-4656-9641-0fa682e1d3c0 None None] MFA Rules not processed for user `a7dc1bc53c59423e934f7831ea022c73`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:03:29.957861 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-68ae182a-b50f-4656-9641-0fa682e1d3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:03:29.965245 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1176/2367] 10.222.0.23 () {58 vars in 967 bytes} [Fri May 6 15:03:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:03:29.978063 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e342f5dd-7956-482b-8314-74a18d355e99 service swift] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:03:29.988674 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e342f5dd-7956-482b-8314-74a18d355e99 service swift] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:03:29.993648 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219212d-360c-4c08-a3b9-8546c5988a6d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf9eed62c8f1409282c2af6488426a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:03:29.994284 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219212d-360c-4c08-a3b9-8546c5988a6d service swift] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:29.994568 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219212d-360c-4c08-a3b9-8546c5988a6d service swift] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:29.994846 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219212d-360c-4c08-a3b9-8546c5988a6d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:30.003768 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0219212d-360c-4c08-a3b9-8546c5988a6d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:03:30.005378 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0219212d-360c-4c08-a3b9-8546c5988a6d service swift] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:03:30.013632 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1192/2368] 10.222.0.23 () {60 vars in 1384 bytes} [Fri May 6 15:03:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 06 15:03:30.559760 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636fffe-e360-4447-814f-a5a6dedf40d9 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:30.560050 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636fffe-e360-4447-814f-a5a6dedf40d9 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:30.560335 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636fffe-e360-4447-814f-a5a6dedf40d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:30.577752 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-f636fffe-e360-4447-814f-a5a6dedf40d9 None None] MFA Rules not processed for user `db19f328e0e448eba77532cdd7bb4c78`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:03:30.578949 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-f636fffe-e360-4447-814f-a5a6dedf40d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:03:30.631735 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1177/2369] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:03:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:03:31.222381 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219212d-360c-4c08-a3b9-8546c5988a6d service swift] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:03:31.286906 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219212d-360c-4c08-a3b9-8546c5988a6d service swift] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:03:31.293125 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c2fa6-7780-404d-89a7-1587ca941843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:03:31.294101 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c2fa6-7780-404d-89a7-1587ca941843 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:31.294683 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c2fa6-7780-404d-89a7-1587ca941843 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:31.295233 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c2fa6-7780-404d-89a7-1587ca941843 admin admin] PATH_INFO: `/v3/users/989a604bec9e4089a54093ef3b9bf5ea` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:31.296471 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618c2fa6-7780-404d-89a7-1587ca941843 admin admin] RBAC: Authorizing `identity:delete_user(user_id=989a604bec9e4089a54093ef3b9bf5ea)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:03:31.298439 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618c2fa6-7780-404d-89a7-1587ca941843 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:03:31.353479 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-618c2fa6-7780-404d-89a7-1587ca941843 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '989a604bec9e4089a54093ef3b9bf5ea'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:03:31.362715 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-618c2fa6-7780-404d-89a7-1587ca941843 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '989a604bec9e4089a54093ef3b9bf5ea'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:03:31.378464 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-618c2fa6-7780-404d-89a7-1587ca941843 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '989a604bec9e4089a54093ef3b9bf5ea'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:03:31.379329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-618c2fa6-7780-404d-89a7-1587ca941843 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '989a604bec9e4089a54093ef3b9bf5ea'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:03:31.380007 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-618c2fa6-7780-404d-89a7-1587ca941843 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '989a604bec9e4089a54093ef3b9bf5ea'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:03:31.400650 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1193/2370] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:03:31 2022] DELETE /identity/v3/users/989a604bec9e4089a54093ef3b9bf5ea => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 15:03:31.406440 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e8610-5e67-4944-8fd5-aac45b0426ea None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:31.409027 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e8610-5e67-4944-8fd5-aac45b0426ea None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:31.409388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e8610-5e67-4944-8fd5-aac45b0426ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:31.428500 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c2fa6-7780-404d-89a7-1587ca941843 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:03:31.430310 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-618c2fa6-7780-404d-89a7-1587ca941843 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:03:31.443246 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-cd5e8610-5e67-4944-8fd5-aac45b0426ea None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:03:31.444454 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-cd5e8610-5e67-4944-8fd5-aac45b0426ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:03:31.509585 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c2fa6-7780-404d-89a7-1587ca941843 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:03:31.518583 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1178/2371] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:03:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:03:31.522684 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe7a216-55d1-4ddd-a819-8a2cafa6b0d7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887a619ef5b5417db98ff15a9f712724', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:03:31.522861 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe7a216-55d1-4ddd-a819-8a2cafa6b0d7 service placement] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:31.523156 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe7a216-55d1-4ddd-a819-8a2cafa6b0d7 service placement] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:31.523463 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe7a216-55d1-4ddd-a819-8a2cafa6b0d7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:31.525501 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-1fe7a216-55d1-4ddd-a819-8a2cafa6b0d7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:03:31.528118 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5e8610-5e67-4944-8fd5-aac45b0426ea None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:03:31.530097 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-cd5e8610-5e67-4944-8fd5-aac45b0426ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:03:31.541606 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe7a216-55d1-4ddd-a819-8a2cafa6b0d7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:03:31.543425 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe7a216-55d1-4ddd-a819-8a2cafa6b0d7 service placement] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:03:31.554947 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1194/2372] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:03:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 15:03:31.609812 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5e8610-5e67-4944-8fd5-aac45b0426ea None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:03:31.618510 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c872ca-0290-42de-81c1-1aab227af8ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:03:31.619185 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c872ca-0290-42de-81c1-1aab227af8ab service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:31.619471 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c872ca-0290-42de-81c1-1aab227af8ab service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:31.619751 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c872ca-0290-42de-81c1-1aab227af8ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:31.643703 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c872ca-0290-42de-81c1-1aab227af8ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:03:31.645440 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c872ca-0290-42de-81c1-1aab227af8ab service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:03:31.661348 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1179/2373] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:03:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 15:03:31.877090 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe7a216-55d1-4ddd-a819-8a2cafa6b0d7 service placement] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:03:31.879110 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-1fe7a216-55d1-4ddd-a819-8a2cafa6b0d7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:03:31.893948 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe7a216-55d1-4ddd-a819-8a2cafa6b0d7 service placement] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:03:31.899112 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0658cb10-2ce5-4959-bed9-e334b87a5dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:03:31.899996 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0658cb10-2ce5-4959-bed9-e334b87a5dd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:31.900283 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0658cb10-2ce5-4959-bed9-e334b87a5dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:31.900564 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0658cb10-2ce5-4959-bed9-e334b87a5dd3 admin admin] PATH_INFO: `/v3/projects/9c9ebfd7aafd4a488325abebd305ffb3` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:31.901841 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0658cb10-2ce5-4959-bed9-e334b87a5dd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c9ebfd7aafd4a488325abebd305ffb3)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:03:31.904555 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0658cb10-2ce5-4959-bed9-e334b87a5dd3 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:03:31.962889 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-0658cb10-2ce5-4959-bed9-e334b87a5dd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c9ebfd7aafd4a488325abebd305ffb3', 'request_id': 'req-0658cb10-2ce5-4959-bed9-e334b87a5dd3'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:03:31.968986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-0658cb10-2ce5-4959-bed9-e334b87a5dd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c9ebfd7aafd4a488325abebd305ffb3', 'request_id': 'req-0658cb10-2ce5-4959-bed9-e334b87a5dd3'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:03:31.976424 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-0658cb10-2ce5-4959-bed9-e334b87a5dd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c9ebfd7aafd4a488325abebd305ffb3', 'request_id': 'req-0658cb10-2ce5-4959-bed9-e334b87a5dd3'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:03:31.988238 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-0658cb10-2ce5-4959-bed9-e334b87a5dd3 admin admin] The token cache is being invalidate because project 9c9ebfd7aafd4a488325abebd305ffb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:03:31.988529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-0658cb10-2ce5-4959-bed9-e334b87a5dd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:03:31.989237 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-0658cb10-2ce5-4959-bed9-e334b87a5dd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:03:31.991071 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1195/2374] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:03:31 2022] DELETE /identity/v3/projects/9c9ebfd7aafd4a488325abebd305ffb3 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:03:31.995815 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039726cc-8a23-4648-9ade-ec76943ccfa6 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:31.996178 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039726cc-8a23-4648-9ade-ec76943ccfa6 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:31.996461 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039726cc-8a23-4648-9ade-ec76943ccfa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:32.016902 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-039726cc-8a23-4648-9ade-ec76943ccfa6 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:03:32.018091 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-039726cc-8a23-4648-9ade-ec76943ccfa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:03:32.079566 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1180/2375] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:03:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 15:03:32.087219 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e7412d-777b-4018-be82-bf2fef5c43b1 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:32.087511 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e7412d-777b-4018-be82-bf2fef5c43b1 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:32.087792 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e7412d-777b-4018-be82-bf2fef5c43b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:32.106864 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-38e7412d-777b-4018-be82-bf2fef5c43b1 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:03:32.108682 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-38e7412d-777b-4018-be82-bf2fef5c43b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:03:32.116813 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1196/2376] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:03:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:03:32.121757 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039726cc-8a23-4648-9ade-ec76943ccfa6 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:03:32.132290 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039726cc-8a23-4648-9ade-ec76943ccfa6 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:03:32.137443 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dece13-f8d2-4493-ba42-bca75bace95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:03:32.138115 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dece13-f8d2-4493-ba42-bca75bace95b admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:32.138406 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dece13-f8d2-4493-ba42-bca75bace95b admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:32.139229 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dece13-f8d2-4493-ba42-bca75bace95b admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:32.139963 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dece13-f8d2-4493-ba42-bca75bace95b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 15:03:32.140273 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dece13-f8d2-4493-ba42-bca75bace95b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:03:32.141921 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dece13-f8d2-4493-ba42-bca75bace95b admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:03:32.156769 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1181/2377] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 15:03:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 15:03:32.161712 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e7412d-777b-4018-be82-bf2fef5c43b1 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:03:32.171986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e7412d-777b-4018-be82-bf2fef5c43b1 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:03:32.177867 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd45f2e2-66c1-4110-8d01-4b3363e5a279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:03:32.179186 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45f2e2-66c1-4110-8d01-4b3363e5a279 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:32.179186 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45f2e2-66c1-4110-8d01-4b3363e5a279 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:32.179186 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45f2e2-66c1-4110-8d01-4b3363e5a279 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:32.180309 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd45f2e2-66c1-4110-8d01-4b3363e5a279 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:03:32.182440 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd45f2e2-66c1-4110-8d01-4b3363e5a279 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:03:32.217472 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1197/2378] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:03:32 2022] POST /identity/v3/projects => generated 391 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:03:32.221688 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dece13-f8d2-4493-ba42-bca75bace95b admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:03:32.237218 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dece13-f8d2-4493-ba42-bca75bace95b admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:03:32.300707 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ac7de-abf8-4e5d-9c04-c170a3b4b988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:03:32.301457 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ac7de-abf8-4e5d-9c04-c170a3b4b988 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:32.302038 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ac7de-abf8-4e5d-9c04-c170a3b4b988 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:32.302354 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ac7de-abf8-4e5d-9c04-c170a3b4b988 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:32.303086 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7ac7de-abf8-4e5d-9c04-c170a3b4b988 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:03:32.305104 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7ac7de-abf8-4e5d-9c04-c170a3b4b988 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:03:32.324908 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1182/2379] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:03:32 2022] POST /identity/v3/users => generated 296 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:03:32.330088 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd45f2e2-66c1-4110-8d01-4b3363e5a279 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:03:32.339940 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd45f2e2-66c1-4110-8d01-4b3363e5a279 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:03:32.346901 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15132801-c482-49e8-942f-de9b3eb53a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:03:32.347570 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15132801-c482-49e8-942f-de9b3eb53a18 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:32.347838 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15132801-c482-49e8-942f-de9b3eb53a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:32.348107 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15132801-c482-49e8-942f-de9b3eb53a18 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:32.348847 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15132801-c482-49e8-942f-de9b3eb53a18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:03:32.350445 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15132801-c482-49e8-942f-de9b3eb53a18 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:03:32.361221 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1198/2380] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:03:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:03:32.365316 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ac7de-abf8-4e5d-9c04-c170a3b4b988 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:03:32.375391 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ac7de-abf8-4e5d-9c04-c170a3b4b988 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:03:32.380109 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb80aa0-7a8c-4e60-9ea3-2b1b3a6204c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:03:32.380811 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb80aa0-7a8c-4e60-9ea3-2b1b3a6204c9 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:32.381097 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb80aa0-7a8c-4e60-9ea3-2b1b3a6204c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:32.381374 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb80aa0-7a8c-4e60-9ea3-2b1b3a6204c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:32.382089 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb80aa0-7a8c-4e60-9ea3-2b1b3a6204c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:03:32.383943 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb80aa0-7a8c-4e60-9ea3-2b1b3a6204c9 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:03:32.395298 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1183/2381] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:03:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:03:32.399597 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15132801-c482-49e8-942f-de9b3eb53a18 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:03:32.411361 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15132801-c482-49e8-942f-de9b3eb53a18 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:03:32.416054 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f16b179-9f4d-4898-b410-8d42cb25f37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:03:32.416990 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16b179-9f4d-4898-b410-8d42cb25f37d admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:32.417268 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16b179-9f4d-4898-b410-8d42cb25f37d admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:32.417537 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16b179-9f4d-4898-b410-8d42cb25f37d admin admin] PATH_INFO: `/v3/projects/a9b53941de4f4a75bcfcc75a1c8c75c6/users/ced6e8a08f854b299bd7131fdef06eb5/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:32.434417 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f16b179-9f4d-4898-b410-8d42cb25f37d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9b53941de4f4a75bcfcc75a1c8c75c6, user_id=ced6e8a08f854b299bd7131fdef06eb5, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:03:32.438232 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f16b179-9f4d-4898-b410-8d42cb25f37d admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:03:32.453985 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1199/2382] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:03:32 2022] PUT /identity/v3/projects/a9b53941de4f4a75bcfcc75a1c8c75c6/users/ced6e8a08f854b299bd7131fdef06eb5/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:03:32.464297 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b45d7-5c8a-4847-943f-ef362bc5ca77 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:32.464868 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b45d7-5c8a-4847-943f-ef362bc5ca77 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:32.464868 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b45d7-5c8a-4847-943f-ef362bc5ca77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:32.495717 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-5e3b45d7-5c8a-4847-943f-ef362bc5ca77 None None] MFA Rules not processed for user `ced6e8a08f854b299bd7131fdef06eb5`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:03:32.496822 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-5e3b45d7-5c8a-4847-943f-ef362bc5ca77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:03:32.575366 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1184/2383] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:03:32 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:03:32.587618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f16b179-9f4d-4898-b410-8d42cb25f37d admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:03:32.589433 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-4f16b179-9f4d-4898-b410-8d42cb25f37d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:03:32.650533 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f16b179-9f4d-4898-b410-8d42cb25f37d admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:03:32.656102 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51768de8-47b3-41e4-8e7a-d0038ccbc25d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:03:32.656861 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51768de8-47b3-41e4-8e7a-d0038ccbc25d service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:32.657168 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51768de8-47b3-41e4-8e7a-d0038ccbc25d service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:32.657435 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51768de8-47b3-41e4-8e7a-d0038ccbc25d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:32.666303 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51768de8-47b3-41e4-8e7a-d0038ccbc25d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:03:32.667728 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51768de8-47b3-41e4-8e7a-d0038ccbc25d service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:03:32.677699 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1200/2384] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:03:32 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:03:36.040961 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b45d7-5c8a-4847-943f-ef362bc5ca77 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:03:36.042541 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-5e3b45d7-5c8a-4847-943f-ef362bc5ca77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:03:36.103564 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b45d7-5c8a-4847-943f-ef362bc5ca77 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:03:36.110094 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca2a37-2bc7-4c21-b4a7-ed855a08c016 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:03:36.110898 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca2a37-2bc7-4c21-b4a7-ed855a08c016 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:36.111281 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca2a37-2bc7-4c21-b4a7-ed855a08c016 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:36.111790 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca2a37-2bc7-4c21-b4a7-ed855a08c016 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:36.113792 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-daca2a37-2bc7-4c21-b4a7-ed855a08c016 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:03:36.125076 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daca2a37-2bc7-4c21-b4a7-ed855a08c016 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:03:36.127296 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daca2a37-2bc7-4c21-b4a7-ed855a08c016 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:03:36.137013 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1185/2385] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:03:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 15:03:39.640302 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51768de8-47b3-41e4-8e7a-d0038ccbc25d service cinder] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:03:39.650372 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51768de8-47b3-41e4-8e7a-d0038ccbc25d service cinder] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:03:39.655469 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8aca89-7577-4dae-9d95-096b275ef73c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:03:39.656345 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8aca89-7577-4dae-9d95-096b275ef73c service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:39.656657 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8aca89-7577-4dae-9d95-096b275ef73c service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:39.656941 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8aca89-7577-4dae-9d95-096b275ef73c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:39.659127 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-fb8aca89-7577-4dae-9d95-096b275ef73c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:03:39.724130 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8aca89-7577-4dae-9d95-096b275ef73c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:03:39.725624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8aca89-7577-4dae-9d95-096b275ef73c service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:03:39.735224 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1201/2386] 10.222.0.23 () {60 vars in 1396 bytes} [Fri May 6 15:03:39 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3242 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:03:43.828257 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca2a37-2bc7-4c21-b4a7-ed855a08c016 service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:03:43.837555 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca2a37-2bc7-4c21-b4a7-ed855a08c016 service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:03:43.841419 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21034d48-c7b0-4db7-a414-31a9a2b7ef3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:03:43.842384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21034d48-c7b0-4db7-a414-31a9a2b7ef3a service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:43.842668 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21034d48-c7b0-4db7-a414-31a9a2b7ef3a service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:43.842942 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21034d48-c7b0-4db7-a414-31a9a2b7ef3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:43.844804 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-21034d48-c7b0-4db7-a414-31a9a2b7ef3a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:03:43.906375 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21034d48-c7b0-4db7-a414-31a9a2b7ef3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:03:43.907806 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21034d48-c7b0-4db7-a414-31a9a2b7ef3a service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:03:43.916764 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1186/2387] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:03:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:03:53.180590 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8aca89-7577-4dae-9d95-096b275ef73c service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:03:53.245167 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8aca89-7577-4dae-9d95-096b275ef73c service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:03:53.250345 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9f8382-e163-43d5-b32d-a38c56a025b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:03:53.251325 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f8382-e163-43d5-b32d-a38c56a025b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:53.251608 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f8382-e163-43d5-b32d-a38c56a025b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:53.251884 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f8382-e163-43d5-b32d-a38c56a025b2 admin admin] PATH_INFO: `/v3/users/ced6e8a08f854b299bd7131fdef06eb5` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:53.253121 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9f8382-e163-43d5-b32d-a38c56a025b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ced6e8a08f854b299bd7131fdef06eb5)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:03:53.255046 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9f8382-e163-43d5-b32d-a38c56a025b2 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:03:53.303465 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-fd9f8382-e163-43d5-b32d-a38c56a025b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ced6e8a08f854b299bd7131fdef06eb5'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:03:53.312126 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-fd9f8382-e163-43d5-b32d-a38c56a025b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ced6e8a08f854b299bd7131fdef06eb5'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:03:53.319439 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-fd9f8382-e163-43d5-b32d-a38c56a025b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ced6e8a08f854b299bd7131fdef06eb5'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:03:53.320084 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-fd9f8382-e163-43d5-b32d-a38c56a025b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ced6e8a08f854b299bd7131fdef06eb5'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:03:53.320824 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-fd9f8382-e163-43d5-b32d-a38c56a025b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ced6e8a08f854b299bd7131fdef06eb5'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:03:53.339949 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1202/2388] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:03:53 2022] DELETE /identity/v3/users/ced6e8a08f854b299bd7131fdef06eb5 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:03:53.345270 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f64dd8a-e717-42ff-9dca-1c0ee0883403 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:53.345772 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f64dd8a-e717-42ff-9dca-1c0ee0883403 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:53.345898 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f64dd8a-e717-42ff-9dca-1c0ee0883403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:53.364379 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-5f64dd8a-e717-42ff-9dca-1c0ee0883403 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:03:53.365689 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-5f64dd8a-e717-42ff-9dca-1c0ee0883403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:03:53.428051 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1187/2389] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:03:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:03:53.439066 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9f8382-e163-43d5-b32d-a38c56a025b2 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:03:53.440510 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-fd9f8382-e163-43d5-b32d-a38c56a025b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:03:53.501194 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9f8382-e163-43d5-b32d-a38c56a025b2 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:03:53.505542 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1167230c-6e85-4250-8212-e707143cf37f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:03:53.506263 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167230c-6e85-4250-8212-e707143cf37f service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:53.506650 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167230c-6e85-4250-8212-e707143cf37f service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:53.507030 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167230c-6e85-4250-8212-e707143cf37f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:53.519507 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1167230c-6e85-4250-8212-e707143cf37f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:03:53.521061 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1167230c-6e85-4250-8212-e707143cf37f service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:03:53.531172 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1203/2390] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:03:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:03:53.793218 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f64dd8a-e717-42ff-9dca-1c0ee0883403 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:03:53.795257 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-5f64dd8a-e717-42ff-9dca-1c0ee0883403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:03:53.810679 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f64dd8a-e717-42ff-9dca-1c0ee0883403 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:03:53.816634 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dec3dea-0314-41d4-ad4a-d12fc1c85220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:03:53.817760 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec3dea-0314-41d4-ad4a-d12fc1c85220 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:53.817928 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec3dea-0314-41d4-ad4a-d12fc1c85220 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:53.818208 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec3dea-0314-41d4-ad4a-d12fc1c85220 admin admin] PATH_INFO: `/v3/projects/a9b53941de4f4a75bcfcc75a1c8c75c6` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:53.819651 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dec3dea-0314-41d4-ad4a-d12fc1c85220 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9b53941de4f4a75bcfcc75a1c8c75c6)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:03:53.821530 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dec3dea-0314-41d4-ad4a-d12fc1c85220 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:03:53.868128 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-3dec3dea-0314-41d4-ad4a-d12fc1c85220 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9b53941de4f4a75bcfcc75a1c8c75c6', 'request_id': 'req-3dec3dea-0314-41d4-ad4a-d12fc1c85220'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:03:53.873377 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-3dec3dea-0314-41d4-ad4a-d12fc1c85220 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9b53941de4f4a75bcfcc75a1c8c75c6', 'request_id': 'req-3dec3dea-0314-41d4-ad4a-d12fc1c85220'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:03:53.880570 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-3dec3dea-0314-41d4-ad4a-d12fc1c85220 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9b53941de4f4a75bcfcc75a1c8c75c6', 'request_id': 'req-3dec3dea-0314-41d4-ad4a-d12fc1c85220'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:03:53.892071 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-3dec3dea-0314-41d4-ad4a-d12fc1c85220 admin admin] The token cache is being invalidate because project a9b53941de4f4a75bcfcc75a1c8c75c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:03:53.892361 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-3dec3dea-0314-41d4-ad4a-d12fc1c85220 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:03:53.893039 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-3dec3dea-0314-41d4-ad4a-d12fc1c85220 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:03:53.894571 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1188/2391] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:03:53 2022] DELETE /identity/v3/projects/a9b53941de4f4a75bcfcc75a1c8c75c6 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:03:53.899501 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830fa20-eb60-4b46-87c3-144b539b520f None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:53.899794 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830fa20-eb60-4b46-87c3-144b539b520f None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:53.900078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830fa20-eb60-4b46-87c3-144b539b520f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:53.918338 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-5830fa20-eb60-4b46-87c3-144b539b520f None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:03:53.919708 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-5830fa20-eb60-4b46-87c3-144b539b520f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:03:53.978332 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1204/2392] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:03:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 15:03:53.987276 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c7b1e-0fcc-4075-ba9d-b9cda247136a None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:53.987570 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c7b1e-0fcc-4075-ba9d-b9cda247136a None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:53.987876 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c7b1e-0fcc-4075-ba9d-b9cda247136a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:54.006116 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-c05c7b1e-0fcc-4075-ba9d-b9cda247136a None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:03:54.007304 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-c05c7b1e-0fcc-4075-ba9d-b9cda247136a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:03:54.015811 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1189/2393] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:03:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:03:54.019958 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5830fa20-eb60-4b46-87c3-144b539b520f None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:03:54.030537 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5830fa20-eb60-4b46-87c3-144b539b520f None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:03:54.035216 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b78f8a-14e8-4212-a056-5ca79780609b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:03:54.035863 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b78f8a-14e8-4212-a056-5ca79780609b admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:54.036145 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b78f8a-14e8-4212-a056-5ca79780609b admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:54.036421 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b78f8a-14e8-4212-a056-5ca79780609b admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:54.037176 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b78f8a-14e8-4212-a056-5ca79780609b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 15:03:54.037483 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b78f8a-14e8-4212-a056-5ca79780609b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:03:54.039059 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b78f8a-14e8-4212-a056-5ca79780609b admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:03:54.050587 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1205/2394] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 15:03:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 15:03:54.054773 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05c7b1e-0fcc-4075-ba9d-b9cda247136a None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:03:54.064679 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05c7b1e-0fcc-4075-ba9d-b9cda247136a None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:03:54.069411 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8444a7dc-cb90-4766-8a99-133722b74c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:03:54.070060 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444a7dc-cb90-4766-8a99-133722b74c67 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:54.070346 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444a7dc-cb90-4766-8a99-133722b74c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:54.070622 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444a7dc-cb90-4766-8a99-133722b74c67 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:54.071340 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8444a7dc-cb90-4766-8a99-133722b74c67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:03:54.073293 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8444a7dc-cb90-4766-8a99-133722b74c67 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:03:54.098592 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1190/2395] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:03:54 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:03:54.102729 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b78f8a-14e8-4212-a056-5ca79780609b admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:03:54.112565 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b78f8a-14e8-4212-a056-5ca79780609b admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:03:54.168648 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5609dbf-75d0-44db-8383-178e40971363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:03:54.169505 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5609dbf-75d0-44db-8383-178e40971363 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:54.169790 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5609dbf-75d0-44db-8383-178e40971363 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:54.170070 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5609dbf-75d0-44db-8383-178e40971363 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:54.170754 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5609dbf-75d0-44db-8383-178e40971363 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:03:54.172696 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5609dbf-75d0-44db-8383-178e40971363 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:03:54.192233 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1206/2396] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:03:54 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 15:03:54.196375 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8444a7dc-cb90-4766-8a99-133722b74c67 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:03:54.207302 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8444a7dc-cb90-4766-8a99-133722b74c67 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:03:54.211826 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4546da35-a4a6-4e96-9180-57a5a6e287b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:03:54.212479 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4546da35-a4a6-4e96-9180-57a5a6e287b6 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:54.212799 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4546da35-a4a6-4e96-9180-57a5a6e287b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:54.213080 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4546da35-a4a6-4e96-9180-57a5a6e287b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:54.213830 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4546da35-a4a6-4e96-9180-57a5a6e287b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:03:54.215424 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4546da35-a4a6-4e96-9180-57a5a6e287b6 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:03:54.226185 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1191/2397] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:03:54.230136 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5609dbf-75d0-44db-8383-178e40971363 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:03:54.239792 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5609dbf-75d0-44db-8383-178e40971363 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:03:54.244312 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f674d-bc8f-48d0-80ea-f2c57f6822ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:03:54.244997 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f674d-bc8f-48d0-80ea-f2c57f6822ad admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:54.245281 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f674d-bc8f-48d0-80ea-f2c57f6822ad admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:54.245550 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f674d-bc8f-48d0-80ea-f2c57f6822ad admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:54.246255 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5f674d-bc8f-48d0-80ea-f2c57f6822ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:03:54.248523 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5f674d-bc8f-48d0-80ea-f2c57f6822ad admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:03:54.258329 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1207/2398] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:03:54.262400 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4546da35-a4a6-4e96-9180-57a5a6e287b6 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:03:54.272738 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4546da35-a4a6-4e96-9180-57a5a6e287b6 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:03:54.277684 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf4f21-7136-42f8-8703-c56d88153f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:03:54.278845 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf4f21-7136-42f8-8703-c56d88153f3e admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:54.279125 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf4f21-7136-42f8-8703-c56d88153f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:54.279403 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf4f21-7136-42f8-8703-c56d88153f3e admin admin] PATH_INFO: `/v3/projects/32f5765bb183428c8a04aed1c66a2068/users/f9f347a369724fcea691938a0391e05e/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:54.297708 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf4f21-7136-42f8-8703-c56d88153f3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=32f5765bb183428c8a04aed1c66a2068, user_id=f9f347a369724fcea691938a0391e05e, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:03:54.300397 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf4f21-7136-42f8-8703-c56d88153f3e admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:03:54.315583 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1192/2399] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:03:54 2022] PUT /identity/v3/projects/32f5765bb183428c8a04aed1c66a2068/users/f9f347a369724fcea691938a0391e05e/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:03:54.320977 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50c1a0-ab0c-4e76-95cb-1834f02dc632 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:54.321299 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50c1a0-ab0c-4e76-95cb-1834f02dc632 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:54.321578 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50c1a0-ab0c-4e76-95cb-1834f02dc632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:54.341066 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-3a50c1a0-ab0c-4e76-95cb-1834f02dc632 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:03:54.342436 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-3a50c1a0-ab0c-4e76-95cb-1834f02dc632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:03:54.406370 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1208/2400] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:03:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:03:54.416285 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf4f21-7136-42f8-8703-c56d88153f3e admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:03:54.418013 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-9ecf4f21-7136-42f8-8703-c56d88153f3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:03:54.475844 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf4f21-7136-42f8-8703-c56d88153f3e admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:03:54.481032 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9148cec9-db69-47e4-b3b4-e8ccb4ff15b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:03:54.482306 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148cec9-db69-47e4-b3b4-e8ccb4ff15b0 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:54.482306 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148cec9-db69-47e4-b3b4-e8ccb4ff15b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:54.482306 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148cec9-db69-47e4-b3b4-e8ccb4ff15b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:54.491720 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9148cec9-db69-47e4-b3b4-e8ccb4ff15b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:03:54.493227 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9148cec9-db69-47e4-b3b4-e8ccb4ff15b0 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:03:54.503148 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1193/2401] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:03:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:03:58.850876 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd7adf-38b5-44d7-8fb4-ba46d2a380d7 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:58.851424 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd7adf-38b5-44d7-8fb4-ba46d2a380d7 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:58.851424 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd7adf-38b5-44d7-8fb4-ba46d2a380d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:58.880401 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-f8bd7adf-38b5-44d7-8fb4-ba46d2a380d7 None None] MFA Rules not processed for user `f9f347a369724fcea691938a0391e05e`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:03:58.882089 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f8bd7adf-38b5-44d7-8fb4-ba46d2a380d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:03:58.956473 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1209/2402] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:03:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:03:58.957408 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9148cec9-db69-47e4-b3b4-e8ccb4ff15b0 service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:03:58.969092 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9148cec9-db69-47e4-b3b4-e8ccb4ff15b0 service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:03:58.974386 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa55e688-66ef-44fa-b4cb-94ebd673a657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:03:58.975385 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55e688-66ef-44fa-b4cb-94ebd673a657 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:58.975666 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55e688-66ef-44fa-b4cb-94ebd673a657 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:58.975938 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55e688-66ef-44fa-b4cb-94ebd673a657 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:58.976648 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa55e688-66ef-44fa-b4cb-94ebd673a657 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:03:58.978583 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa55e688-66ef-44fa-b4cb-94ebd673a657 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:03:59.006317 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1194/2403] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:03:58 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 15:03:59.010139 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bd7adf-38b5-44d7-8fb4-ba46d2a380d7 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:03:59.022360 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bd7adf-38b5-44d7-8fb4-ba46d2a380d7 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:03:59.082327 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f94d5-22e8-4a7d-b08c-b4234f8be2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:03:59.083008 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f94d5-22e8-4a7d-b08c-b4234f8be2ec admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:59.083290 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f94d5-22e8-4a7d-b08c-b4234f8be2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:59.083566 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f94d5-22e8-4a7d-b08c-b4234f8be2ec admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:59.084287 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3f94d5-22e8-4a7d-b08c-b4234f8be2ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:03:59.086246 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3f94d5-22e8-4a7d-b08c-b4234f8be2ec admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:03:59.109612 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1210/2404] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:03:59 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:03:59.113652 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa55e688-66ef-44fa-b4cb-94ebd673a657 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:03:59.125019 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa55e688-66ef-44fa-b4cb-94ebd673a657 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:03:59.129807 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c6552-dc1d-4fab-bfc5-a0360fe0e7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:03:59.130470 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c6552-dc1d-4fab-bfc5-a0360fe0e7c2 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:59.130748 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c6552-dc1d-4fab-bfc5-a0360fe0e7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:59.131029 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c6552-dc1d-4fab-bfc5-a0360fe0e7c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:59.131737 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c6552-dc1d-4fab-bfc5-a0360fe0e7c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:03:59.133372 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c6552-dc1d-4fab-bfc5-a0360fe0e7c2 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:03:59.146763 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1195/2405] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:03:59.154684 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f94d5-22e8-4a7d-b08c-b4234f8be2ec admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:03:59.165901 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f94d5-22e8-4a7d-b08c-b4234f8be2ec admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:03:59.171482 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af843a0e-cf47-482d-9a14-12d42f676b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:03:59.172364 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af843a0e-cf47-482d-9a14-12d42f676b8e admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:59.172758 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af843a0e-cf47-482d-9a14-12d42f676b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:59.173036 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af843a0e-cf47-482d-9a14-12d42f676b8e admin admin] PATH_INFO: `/v3/projects/a62c7d504fd44db9a30bea8d201735c6/users/9af2da9ba76e48b497c16cb152eaa8a5/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:59.188611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af843a0e-cf47-482d-9a14-12d42f676b8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a62c7d504fd44db9a30bea8d201735c6, user_id=9af2da9ba76e48b497c16cb152eaa8a5, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:03:59.191272 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af843a0e-cf47-482d-9a14-12d42f676b8e admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:03:59.204051 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1211/2406] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:03:59 2022] PUT /identity/v3/projects/a62c7d504fd44db9a30bea8d201735c6/users/9af2da9ba76e48b497c16cb152eaa8a5/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 15:03:59.208036 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c6552-dc1d-4fab-bfc5-a0360fe0e7c2 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:03:59.218462 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c6552-dc1d-4fab-bfc5-a0360fe0e7c2 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:03:59.274374 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c420675-2c29-4826-885f-5d2c44057292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:03:59.275146 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c420675-2c29-4826-885f-5d2c44057292 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:59.275426 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c420675-2c29-4826-885f-5d2c44057292 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:59.275696 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c420675-2c29-4826-885f-5d2c44057292 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:59.276430 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c420675-2c29-4826-885f-5d2c44057292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:03:59.278888 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c420675-2c29-4826-885f-5d2c44057292 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:03:59.288506 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1196/2407] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:03:59.292720 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af843a0e-cf47-482d-9a14-12d42f676b8e admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:03:59.304108 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af843a0e-cf47-482d-9a14-12d42f676b8e admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:03:59.312965 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379808f-1bc0-4809-ad90-323df97e15c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:03:59.314269 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379808f-1bc0-4809-ad90-323df97e15c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:03:59.314554 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379808f-1bc0-4809-ad90-323df97e15c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:03:59.314856 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379808f-1bc0-4809-ad90-323df97e15c0 admin admin] PATH_INFO: `/v3/projects/a62c7d504fd44db9a30bea8d201735c6/users/9af2da9ba76e48b497c16cb152eaa8a5/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:03:59.316894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0379808f-1bc0-4809-ad90-323df97e15c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a62c7d504fd44db9a30bea8d201735c6, user_id=9af2da9ba76e48b497c16cb152eaa8a5, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:03:59.319803 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0379808f-1bc0-4809-ad90-323df97e15c0 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:03:59.333043 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1212/2408] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:03:59 2022] PUT /identity/v3/projects/a62c7d504fd44db9a30bea8d201735c6/users/9af2da9ba76e48b497c16cb152eaa8a5/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:04:04.526058 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d26f9c-0826-411d-a85a-a9487a6b2f0b None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:04:04.527179 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d26f9c-0826-411d-a85a-a9487a6b2f0b None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:04:04.527458 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d26f9c-0826-411d-a85a-a9487a6b2f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:04:04.560115 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-a5d26f9c-0826-411d-a85a-a9487a6b2f0b None None] MFA Rules not processed for user `9af2da9ba76e48b497c16cb152eaa8a5`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:04:04.561383 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-a5d26f9c-0826-411d-a85a-a9487a6b2f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:04:04.639482 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1197/2409] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:04:04 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:04:04.650622 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379808f-1bc0-4809-ad90-323df97e15c0 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:04:04.652441 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-0379808f-1bc0-4809-ad90-323df97e15c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:04:04.732854 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379808f-1bc0-4809-ad90-323df97e15c0 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:04:04.738387 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7faa9ef-8454-4a72-bea4-51dc12f0dc12 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:04:04.739064 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7faa9ef-8454-4a72-bea4-51dc12f0dc12 service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:04:04.739343 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7faa9ef-8454-4a72-bea4-51dc12f0dc12 service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:04:04.739617 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7faa9ef-8454-4a72-bea4-51dc12f0dc12 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:04:04.799561 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7faa9ef-8454-4a72-bea4-51dc12f0dc12 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:04:04.801201 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7faa9ef-8454-4a72-bea4-51dc12f0dc12 service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:04:04.811220 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1213/2410] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:04:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:04:06.276630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a8e87-4b41-4d21-91ae-2e0400eec309 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:04:06.277275 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a8e87-4b41-4d21-91ae-2e0400eec309 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:04:06.277381 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a8e87-4b41-4d21-91ae-2e0400eec309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:04:06.299308 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-ba5a8e87-4b41-4d21-91ae-2e0400eec309 None None] MFA Rules not processed for user `f9f347a369724fcea691938a0391e05e`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:04:06.300834 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ba5a8e87-4b41-4d21-91ae-2e0400eec309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:04:06.308430 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1198/2411] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:04:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:04:23.362443 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7faa9ef-8454-4a72-bea4-51dc12f0dc12 service nova] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:04:23.363915 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-a7faa9ef-8454-4a72-bea4-51dc12f0dc12 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:04:23.378769 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7faa9ef-8454-4a72-bea4-51dc12f0dc12 service nova] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:04:23.384357 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cd832d-0055-4379-953a-72c200df13ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:04:23.385042 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd832d-0055-4379-953a-72c200df13ff service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:04:23.385322 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd832d-0055-4379-953a-72c200df13ff service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:04:23.385601 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd832d-0055-4379-953a-72c200df13ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:04:23.387993 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e0cd832d-0055-4379-953a-72c200df13ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:04:23.400857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cd832d-0055-4379-953a-72c200df13ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:04:23.402574 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cd832d-0055-4379-953a-72c200df13ff service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:04:23.416007 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1214/2412] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:04:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:04:26.462493 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff125b-7f27-4f7b-a865-a50c0161ef2d None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:04:26.463058 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff125b-7f27-4f7b-a865-a50c0161ef2d None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:04:26.463058 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff125b-7f27-4f7b-a865-a50c0161ef2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:04:26.481319 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-51ff125b-7f27-4f7b-a865-a50c0161ef2d None None] MFA Rules not processed for user `81c9e87c348c4a2c82e8327a2cadfc5d`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:04:26.482535 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-51ff125b-7f27-4f7b-a865-a50c0161ef2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:04:26.537122 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1199/2413] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:04:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:04:36.644551 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cd832d-0055-4379-953a-72c200df13ff service nova] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:04:36.704197 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cd832d-0055-4379-953a-72c200df13ff service nova] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:04:36.707801 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b59ea-668d-4e9c-a6fb-ea7e5789e5b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:04:36.708476 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b59ea-668d-4e9c-a6fb-ea7e5789e5b6 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:04:36.708916 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b59ea-668d-4e9c-a6fb-ea7e5789e5b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:04:36.709202 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b59ea-668d-4e9c-a6fb-ea7e5789e5b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:04:36.712204 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-820b59ea-668d-4e9c-a6fb-ea7e5789e5b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:04:36.722779 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820b59ea-668d-4e9c-a6fb-ea7e5789e5b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:04:36.724200 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820b59ea-668d-4e9c-a6fb-ea7e5789e5b6 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:04:36.732436 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1215/2414] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:04:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 15:04:37.692390 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ff125b-7f27-4f7b-a865-a50c0161ef2d None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:04:37.717791 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ff125b-7f27-4f7b-a865-a50c0161ef2d None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:04:37.723956 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70309c0-1a34-4e07-943a-cb57226b5d85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:04:37.724816 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70309c0-1a34-4e07-943a-cb57226b5d85 service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:04:37.725201 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70309c0-1a34-4e07-943a-cb57226b5d85 service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:04:37.725550 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70309c0-1a34-4e07-943a-cb57226b5d85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:04:37.727518 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-a70309c0-1a34-4e07-943a-cb57226b5d85 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:04:37.742728 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70309c0-1a34-4e07-943a-cb57226b5d85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:04:37.744516 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70309c0-1a34-4e07-943a-cb57226b5d85 service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:04:37.757313 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1200/2415] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:04:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:04:51.725785 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b59ea-668d-4e9c-a6fb-ea7e5789e5b6 service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:04:51.728020 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-820b59ea-668d-4e9c-a6fb-ea7e5789e5b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:04:51.798359 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b59ea-668d-4e9c-a6fb-ea7e5789e5b6 service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:04:51.803353 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48aa28d-1090-43d4-bdfc-d98045abcb01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:04:51.804220 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48aa28d-1090-43d4-bdfc-d98045abcb01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:04:51.804499 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48aa28d-1090-43d4-bdfc-d98045abcb01 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:04:51.804851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48aa28d-1090-43d4-bdfc-d98045abcb01 admin admin] PATH_INFO: `/v3/users/db19f328e0e448eba77532cdd7bb4c78` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:04:51.806038 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48aa28d-1090-43d4-bdfc-d98045abcb01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db19f328e0e448eba77532cdd7bb4c78)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:04:51.809006 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48aa28d-1090-43d4-bdfc-d98045abcb01 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:04:51.862157 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d48aa28d-1090-43d4-bdfc-d98045abcb01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db19f328e0e448eba77532cdd7bb4c78'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:04:51.870618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d48aa28d-1090-43d4-bdfc-d98045abcb01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db19f328e0e448eba77532cdd7bb4c78'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:04:51.878208 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d48aa28d-1090-43d4-bdfc-d98045abcb01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db19f328e0e448eba77532cdd7bb4c78'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:04:51.879493 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d48aa28d-1090-43d4-bdfc-d98045abcb01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db19f328e0e448eba77532cdd7bb4c78'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:04:51.880112 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d48aa28d-1090-43d4-bdfc-d98045abcb01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db19f328e0e448eba77532cdd7bb4c78'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:04:51.902456 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1216/2416] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:04:51 2022] DELETE /identity/v3/users/db19f328e0e448eba77532cdd7bb4c78 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:04:51.902919 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70309c0-1a34-4e07-943a-cb57226b5d85 service nova] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:04:51.914498 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70309c0-1a34-4e07-943a-cb57226b5d85 service nova] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:04:52.012502 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e32de7-b3ed-4647-8b12-75b2fa84b585 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:04:52.013218 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e32de7-b3ed-4647-8b12-75b2fa84b585 service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:04:52.013503 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e32de7-b3ed-4647-8b12-75b2fa84b585 service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:04:52.013784 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e32de7-b3ed-4647-8b12-75b2fa84b585 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:04:52.015674 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-35e32de7-b3ed-4647-8b12-75b2fa84b585 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:04:52.085758 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e32de7-b3ed-4647-8b12-75b2fa84b585 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:04:52.087403 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e32de7-b3ed-4647-8b12-75b2fa84b585 service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:04:52.098769 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1201/2417] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:04:51 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 15:04:52.106873 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48aa28d-1090-43d4-bdfc-d98045abcb01 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:04:52.110150 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-d48aa28d-1090-43d4-bdfc-d98045abcb01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:04:52.183805 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48aa28d-1090-43d4-bdfc-d98045abcb01 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:04:52.188622 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d94d9a3-29a5-4e8a-8d16-4902d9c79560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:04:52.189504 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94d9a3-29a5-4e8a-8d16-4902d9c79560 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:04:52.189790 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94d9a3-29a5-4e8a-8d16-4902d9c79560 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:04:52.190067 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94d9a3-29a5-4e8a-8d16-4902d9c79560 admin admin] PATH_INFO: `/v3/projects/d266c6c15e474b7885134dbe78a29833` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:04:52.191627 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d94d9a3-29a5-4e8a-8d16-4902d9c79560 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d266c6c15e474b7885134dbe78a29833)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:04:52.193632 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d94d9a3-29a5-4e8a-8d16-4902d9c79560 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:04:52.253895 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9d94d9a3-29a5-4e8a-8d16-4902d9c79560 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd266c6c15e474b7885134dbe78a29833', 'request_id': 'req-9d94d9a3-29a5-4e8a-8d16-4902d9c79560'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:04:52.259242 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9d94d9a3-29a5-4e8a-8d16-4902d9c79560 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd266c6c15e474b7885134dbe78a29833', 'request_id': 'req-9d94d9a3-29a5-4e8a-8d16-4902d9c79560'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:04:52.273017 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9d94d9a3-29a5-4e8a-8d16-4902d9c79560 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd266c6c15e474b7885134dbe78a29833', 'request_id': 'req-9d94d9a3-29a5-4e8a-8d16-4902d9c79560'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:04:52.286467 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9d94d9a3-29a5-4e8a-8d16-4902d9c79560 admin admin] The token cache is being invalidate because project d266c6c15e474b7885134dbe78a29833 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:04:52.286893 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9d94d9a3-29a5-4e8a-8d16-4902d9c79560 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:04:52.287563 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9d94d9a3-29a5-4e8a-8d16-4902d9c79560 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:04:52.289161 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1217/2418] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:04:52 2022] DELETE /identity/v3/projects/d266c6c15e474b7885134dbe78a29833 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 15:04:54.739912 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e32de7-b3ed-4647-8b12-75b2fa84b585 service nova] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:04:54.742234 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-35e32de7-b3ed-4647-8b12-75b2fa84b585 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:04:54.757347 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e32de7-b3ed-4647-8b12-75b2fa84b585 service nova] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:04:54.833992 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df815ec1-60c1-41a7-8005-420d2fa84a68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:04:54.834588 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df815ec1-60c1-41a7-8005-420d2fa84a68 service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:04:54.834867 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df815ec1-60c1-41a7-8005-420d2fa84a68 service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:04:54.835139 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df815ec1-60c1-41a7-8005-420d2fa84a68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:04:54.836938 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-df815ec1-60c1-41a7-8005-420d2fa84a68 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:04:54.849722 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df815ec1-60c1-41a7-8005-420d2fa84a68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:04:54.851794 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df815ec1-60c1-41a7-8005-420d2fa84a68 service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:04:54.864160 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1202/2419] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:04:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:04:59.908590 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d94d9a3-29a5-4e8a-8d16-4902d9c79560 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:04:59.910172 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-9d94d9a3-29a5-4e8a-8d16-4902d9c79560 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:04:59.970252 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d94d9a3-29a5-4e8a-8d16-4902d9c79560 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:04:59.976488 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf55b7-4c87-4f7b-9beb-8e5dd27a55bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:04:59.977768 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf55b7-4c87-4f7b-9beb-8e5dd27a55bf service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:04:59.977768 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf55b7-4c87-4f7b-9beb-8e5dd27a55bf service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:04:59.977768 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf55b7-4c87-4f7b-9beb-8e5dd27a55bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:04:59.979949 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-b7cf55b7-4c87-4f7b-9beb-8e5dd27a55bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:05:00.052186 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf55b7-4c87-4f7b-9beb-8e5dd27a55bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:05:00.054974 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf55b7-4c87-4f7b-9beb-8e5dd27a55bf service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:05:00.073607 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1218/2420] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:04:59 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:05:01.913875 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df815ec1-60c1-41a7-8005-420d2fa84a68 service nova] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:05:01.916359 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-df815ec1-60c1-41a7-8005-420d2fa84a68 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:05:01.937865 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df815ec1-60c1-41a7-8005-420d2fa84a68 service nova] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:05:01.945437 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db9a9f7-23cf-48f5-91ec-58db65c1953e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:05:01.946481 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db9a9f7-23cf-48f5-91ec-58db65c1953e service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:05:01.947257 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db9a9f7-23cf-48f5-91ec-58db65c1953e service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:05:01.947551 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db9a9f7-23cf-48f5-91ec-58db65c1953e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:05:01.951447 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-5db9a9f7-23cf-48f5-91ec-58db65c1953e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:05:02.066638 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db9a9f7-23cf-48f5-91ec-58db65c1953e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:05:02.069365 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db9a9f7-23cf-48f5-91ec-58db65c1953e service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:05:02.099465 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1203/2421] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:05:01 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:05:02.136595 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d8231-ec80-4e64-9242-3ad82c181eaa None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:05:02.136595 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d8231-ec80-4e64-9242-3ad82c181eaa None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:05:02.137206 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d8231-ec80-4e64-9242-3ad82c181eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:05:02.171940 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-fd4d8231-ec80-4e64-9242-3ad82c181eaa None None] MFA Rules not processed for user `4b3af1d82c33446e998f07bce54ef941`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:05:02.175669 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-fd4d8231-ec80-4e64-9242-3ad82c181eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:05:02.208777 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1219/2422] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:05:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:05:02.588933 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db9a9f7-23cf-48f5-91ec-58db65c1953e service nova] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:05:02.599796 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db9a9f7-23cf-48f5-91ec-58db65c1953e service nova] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:05:02.603927 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb76f67b-ae01-42e8-be71-0f866ac1d78a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:05:02.604690 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb76f67b-ae01-42e8-be71-0f866ac1d78a service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:05:02.605170 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb76f67b-ae01-42e8-be71-0f866ac1d78a service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:05:02.605514 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb76f67b-ae01-42e8-be71-0f866ac1d78a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:05:02.608625 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-cb76f67b-ae01-42e8-be71-0f866ac1d78a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:05:02.624259 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb76f67b-ae01-42e8-be71-0f866ac1d78a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:05:02.628531 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb76f67b-ae01-42e8-be71-0f866ac1d78a service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:05:02.650791 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1204/2423] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:05:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 15:05:06.009234 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd02a68-6540-4b83-bd54-eb6c30e815ae None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:05:06.009754 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd02a68-6540-4b83-bd54-eb6c30e815ae None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:05:06.009871 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd02a68-6540-4b83-bd54-eb6c30e815ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:05:06.027308 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-0dd02a68-6540-4b83-bd54-eb6c30e815ae None None] MFA Rules not processed for user `02b07e776b2c4bfba6eabb15999aec35`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:05:06.028799 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-0dd02a68-6540-4b83-bd54-eb6c30e815ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:05:06.083632 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1220/2424] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:05:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:05:09.949244 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee453e3-36a9-481a-8b36-c9fa3874f14a None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:05:09.949865 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee453e3-36a9-481a-8b36-c9fa3874f14a None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:05:09.949865 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee453e3-36a9-481a-8b36-c9fa3874f14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:05:09.969225 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-7ee453e3-36a9-481a-8b36-c9fa3874f14a None None] MFA Rules not processed for user `afd9f5519134415b8fc8a8c08149c502`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:05:09.970415 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-7ee453e3-36a9-481a-8b36-c9fa3874f14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:05:10.028249 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1205/2425] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:05:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:05:54.249431 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd02a68-6540-4b83-bd54-eb6c30e815ae None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:05:54.258942 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd02a68-6540-4b83-bd54-eb6c30e815ae None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:05:54.262476 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd6aff-5f9e-49a9-9013-5762e0f51f2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:05:54.263159 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd6aff-5f9e-49a9-9013-5762e0f51f2d service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:05:54.263445 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd6aff-5f9e-49a9-9013-5762e0f51f2d service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:05:54.263723 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd6aff-5f9e-49a9-9013-5762e0f51f2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:05:54.265489 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-84bd6aff-5f9e-49a9-9013-5762e0f51f2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:05:54.327327 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bd6aff-5f9e-49a9-9013-5762e0f51f2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:05:54.328822 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bd6aff-5f9e-49a9-9013-5762e0f51f2d service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:05:54.339188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1221/2426] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:05:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:05:57.722067 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee453e3-36a9-481a-8b36-c9fa3874f14a None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:05:57.724556 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-7ee453e3-36a9-481a-8b36-c9fa3874f14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:05:57.742690 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee453e3-36a9-481a-8b36-c9fa3874f14a None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:05:57.747287 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056b9f34-2989-4740-9622-be5bc07b0c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:05:57.748164 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b9f34-2989-4740-9622-be5bc07b0c14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:05:57.748445 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b9f34-2989-4740-9622-be5bc07b0c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:05:57.748758 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b9f34-2989-4740-9622-be5bc07b0c14 admin admin] PATH_INFO: `/v3/users/81c9e87c348c4a2c82e8327a2cadfc5d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:05:57.750007 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056b9f34-2989-4740-9622-be5bc07b0c14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81c9e87c348c4a2c82e8327a2cadfc5d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:05:57.751975 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056b9f34-2989-4740-9622-be5bc07b0c14 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:05:57.811912 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-056b9f34-2989-4740-9622-be5bc07b0c14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81c9e87c348c4a2c82e8327a2cadfc5d'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:05:57.812701 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-056b9f34-2989-4740-9622-be5bc07b0c14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81c9e87c348c4a2c82e8327a2cadfc5d'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:05:57.820675 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-056b9f34-2989-4740-9622-be5bc07b0c14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81c9e87c348c4a2c82e8327a2cadfc5d'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:05:57.828040 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-056b9f34-2989-4740-9622-be5bc07b0c14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81c9e87c348c4a2c82e8327a2cadfc5d'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:05:57.828851 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-056b9f34-2989-4740-9622-be5bc07b0c14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81c9e87c348c4a2c82e8327a2cadfc5d'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:05:57.846284 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1206/2427] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:05:57 2022] DELETE /identity/v3/users/81c9e87c348c4a2c82e8327a2cadfc5d => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:05:58.027999 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd6aff-5f9e-49a9-9013-5762e0f51f2d service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:05:58.043845 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd6aff-5f9e-49a9-9013-5762e0f51f2d service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:05:58.120008 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67147bab-0fc2-4766-bacf-3cb1375d98f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:05:58.121356 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67147bab-0fc2-4766-bacf-3cb1375d98f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:05:58.121973 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67147bab-0fc2-4766-bacf-3cb1375d98f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:05:58.122321 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67147bab-0fc2-4766-bacf-3cb1375d98f3 admin admin] PATH_INFO: `/v3/projects/66121d02dbff4c3a920161e445959085` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:05:58.123692 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67147bab-0fc2-4766-bacf-3cb1375d98f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66121d02dbff4c3a920161e445959085)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:05:58.125792 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67147bab-0fc2-4766-bacf-3cb1375d98f3 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:05:58.199287 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-67147bab-0fc2-4766-bacf-3cb1375d98f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66121d02dbff4c3a920161e445959085', 'request_id': 'req-67147bab-0fc2-4766-bacf-3cb1375d98f3'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:05:58.205511 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-67147bab-0fc2-4766-bacf-3cb1375d98f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66121d02dbff4c3a920161e445959085', 'request_id': 'req-67147bab-0fc2-4766-bacf-3cb1375d98f3'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:05:58.216880 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-67147bab-0fc2-4766-bacf-3cb1375d98f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66121d02dbff4c3a920161e445959085', 'request_id': 'req-67147bab-0fc2-4766-bacf-3cb1375d98f3'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:05:58.229748 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-67147bab-0fc2-4766-bacf-3cb1375d98f3 admin admin] The token cache is being invalidate because project 66121d02dbff4c3a920161e445959085 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:05:58.230044 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-67147bab-0fc2-4766-bacf-3cb1375d98f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:05:58.230731 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-67147bab-0fc2-4766-bacf-3cb1375d98f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:05:58.232346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1222/2428] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:05:58 2022] DELETE /identity/v3/projects/66121d02dbff4c3a920161e445959085 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:05:58.237829 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c21edb-f812-4c08-945a-cc08f233a8b9 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:05:58.238128 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c21edb-f812-4c08-945a-cc08f233a8b9 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:05:58.238411 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c21edb-f812-4c08-945a-cc08f233a8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:05:58.258442 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-b8c21edb-f812-4c08-945a-cc08f233a8b9 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:05:58.259740 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b8c21edb-f812-4c08-945a-cc08f233a8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:05:58.320290 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1207/2429] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:05:58.328298 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675bcb1-c849-4a4c-94f3-096972d94cd8 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:05:58.328674 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675bcb1-c849-4a4c-94f3-096972d94cd8 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:05:58.328960 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675bcb1-c849-4a4c-94f3-096972d94cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:05:58.348080 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-a675bcb1-c849-4a4c-94f3-096972d94cd8 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:05:58.349303 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-a675bcb1-c849-4a4c-94f3-096972d94cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:05:58.357858 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1223/2430] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 15:05:58.362107 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c21edb-f812-4c08-945a-cc08f233a8b9 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:05:58.379906 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c21edb-f812-4c08-945a-cc08f233a8b9 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:05:58.384755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c8175d-6048-4aff-9c97-1319269d6073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:05:58.385422 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8175d-6048-4aff-9c97-1319269d6073 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:05:58.385704 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8175d-6048-4aff-9c97-1319269d6073 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:05:58.385980 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8175d-6048-4aff-9c97-1319269d6073 admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:05:58.386733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c8175d-6048-4aff-9c97-1319269d6073 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 15:05:58.387063 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c8175d-6048-4aff-9c97-1319269d6073 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:05:58.388707 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c8175d-6048-4aff-9c97-1319269d6073 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:05:58.400538 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1208/2431] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 15:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 15:05:58.405256 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a675bcb1-c849-4a4c-94f3-096972d94cd8 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:05:58.415314 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a675bcb1-c849-4a4c-94f3-096972d94cd8 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:05:58.420077 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b507a85-672e-435b-b675-3752e0df918b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:05:58.420786 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b507a85-672e-435b-b675-3752e0df918b admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:05:58.421072 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b507a85-672e-435b-b675-3752e0df918b admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:05:58.421348 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b507a85-672e-435b-b675-3752e0df918b admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:05:58.422033 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b507a85-672e-435b-b675-3752e0df918b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:05:58.424136 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b507a85-672e-435b-b675-3752e0df918b admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:05:58.459837 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1224/2432] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:05:58 2022] POST /identity/v3/projects => generated 403 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:05:58.464392 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c8175d-6048-4aff-9c97-1319269d6073 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:05:58.474236 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c8175d-6048-4aff-9c97-1319269d6073 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:05:58.528733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772dff74-ccf1-4c0e-b055-0bfe4091062c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:05:58.529316 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772dff74-ccf1-4c0e-b055-0bfe4091062c admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:05:58.529598 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772dff74-ccf1-4c0e-b055-0bfe4091062c admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:05:58.529877 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772dff74-ccf1-4c0e-b055-0bfe4091062c admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:05:58.530558 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772dff74-ccf1-4c0e-b055-0bfe4091062c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:05:58.532473 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772dff74-ccf1-4c0e-b055-0bfe4091062c admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:05:58.556131 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1209/2433] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:05:58 2022] POST /identity/v3/users => generated 302 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:05:58.557191 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b507a85-672e-435b-b675-3752e0df918b admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:05:58.567372 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b507a85-672e-435b-b675-3752e0df918b admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:05:58.572252 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597a91ed-d37a-402a-9aa2-f16aa7e992aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:05:58.572943 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a91ed-d37a-402a-9aa2-f16aa7e992aa admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:05:58.573226 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a91ed-d37a-402a-9aa2-f16aa7e992aa admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:05:58.573499 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a91ed-d37a-402a-9aa2-f16aa7e992aa admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:05:58.574222 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597a91ed-d37a-402a-9aa2-f16aa7e992aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:05:58.575832 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597a91ed-d37a-402a-9aa2-f16aa7e992aa admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:05:58.586566 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1225/2434] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:05:58.591529 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772dff74-ccf1-4c0e-b055-0bfe4091062c admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:05:58.602177 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772dff74-ccf1-4c0e-b055-0bfe4091062c admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:05:58.606917 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce8b8f4-2ec1-4780-833b-1ad42a6aeb5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:05:58.607584 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce8b8f4-2ec1-4780-833b-1ad42a6aeb5d admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:05:58.607865 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce8b8f4-2ec1-4780-833b-1ad42a6aeb5d admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:05:58.608142 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce8b8f4-2ec1-4780-833b-1ad42a6aeb5d admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:05:58.608892 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce8b8f4-2ec1-4780-833b-1ad42a6aeb5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:05:58.610475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce8b8f4-2ec1-4780-833b-1ad42a6aeb5d admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:05:58.621127 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1210/2435] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:05:58.625428 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597a91ed-d37a-402a-9aa2-f16aa7e992aa admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:05:58.635542 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597a91ed-d37a-402a-9aa2-f16aa7e992aa admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:05:58.640191 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c462aa37-4152-4680-a7d1-271bc7b796dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:05:58.641101 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462aa37-4152-4680-a7d1-271bc7b796dd admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:05:58.641392 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462aa37-4152-4680-a7d1-271bc7b796dd admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:05:58.641668 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462aa37-4152-4680-a7d1-271bc7b796dd admin admin] PATH_INFO: `/v3/projects/837aa9d1761e4608bc8300c52c1db311/users/bbc1dc9335824dcba4c1d2c25dfb8684/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:05:58.657099 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c462aa37-4152-4680-a7d1-271bc7b796dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=837aa9d1761e4608bc8300c52c1db311, user_id=bbc1dc9335824dcba4c1d2c25dfb8684, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:05:58.659753 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c462aa37-4152-4680-a7d1-271bc7b796dd admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:05:58.674093 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1226/2436] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:05:58 2022] PUT /identity/v3/projects/837aa9d1761e4608bc8300c52c1db311/users/bbc1dc9335824dcba4c1d2c25dfb8684/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:05:58.685700 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798129c5-3fd4-4f26-ac27-713c4420ade1 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:05:58.685994 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798129c5-3fd4-4f26-ac27-713c4420ade1 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:05:58.686277 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798129c5-3fd4-4f26-ac27-713c4420ade1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:05:58.714074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-798129c5-3fd4-4f26-ac27-713c4420ade1 None None] MFA Rules not processed for user `bbc1dc9335824dcba4c1d2c25dfb8684`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:05:58.715311 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-798129c5-3fd4-4f26-ac27-713c4420ade1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:05:58.777683 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1211/2437] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:05:58 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:05:58.782217 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c462aa37-4152-4680-a7d1-271bc7b796dd admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:05:58.841904 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c462aa37-4152-4680-a7d1-271bc7b796dd admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:05:58.846618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85008f5-df8b-47de-af10-5749a4a82761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:05:58.847281 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85008f5-df8b-47de-af10-5749a4a82761 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:05:58.847569 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85008f5-df8b-47de-af10-5749a4a82761 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:05:58.847853 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85008f5-df8b-47de-af10-5749a4a82761 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:05:58.848550 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85008f5-df8b-47de-af10-5749a4a82761 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:05:58.851106 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85008f5-df8b-47de-af10-5749a4a82761 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:05:58.876657 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1227/2438] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:05:58 2022] POST /identity/v3/projects => generated 405 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:05:58.880385 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798129c5-3fd4-4f26-ac27-713c4420ade1 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:05:58.891183 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798129c5-3fd4-4f26-ac27-713c4420ade1 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:05:58.947632 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f6b32-bfa0-4bf4-9ec6-63b416e48429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:05:58.948302 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f6b32-bfa0-4bf4-9ec6-63b416e48429 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:05:58.948582 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f6b32-bfa0-4bf4-9ec6-63b416e48429 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:05:58.948900 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f6b32-bfa0-4bf4-9ec6-63b416e48429 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:05:58.949596 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14f6b32-bfa0-4bf4-9ec6-63b416e48429 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:05:58.951510 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14f6b32-bfa0-4bf4-9ec6-63b416e48429 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:05:58.971589 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1212/2439] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:05:58 2022] POST /identity/v3/users => generated 309 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 15:05:58.975722 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85008f5-df8b-47de-af10-5749a4a82761 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:05:58.986044 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85008f5-df8b-47de-af10-5749a4a82761 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:05:58.990695 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917bc816-5836-4859-a7a3-53de231604d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:05:58.991479 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917bc816-5836-4859-a7a3-53de231604d4 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:05:58.991782 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917bc816-5836-4859-a7a3-53de231604d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:05:58.992060 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917bc816-5836-4859-a7a3-53de231604d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:05:58.992832 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917bc816-5836-4859-a7a3-53de231604d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:05:58.994407 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917bc816-5836-4859-a7a3-53de231604d4 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:05:59.005191 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1228/2440] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:05:59.010554 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f6b32-bfa0-4bf4-9ec6-63b416e48429 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:05:59.020757 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f6b32-bfa0-4bf4-9ec6-63b416e48429 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:05:59.026135 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68061d1b-1c21-42dd-bc34-e320208f2caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:05:59.027412 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68061d1b-1c21-42dd-bc34-e320208f2caa admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:05:59.027694 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68061d1b-1c21-42dd-bc34-e320208f2caa admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:05:59.027971 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68061d1b-1c21-42dd-bc34-e320208f2caa admin admin] PATH_INFO: `/v3/projects/7f9817356e674c9c8bae5825a3ca9ce2/users/137fb55cf6e24c03a740e88e24694704/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:05:59.043008 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68061d1b-1c21-42dd-bc34-e320208f2caa admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f9817356e674c9c8bae5825a3ca9ce2, user_id=137fb55cf6e24c03a740e88e24694704, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:05:59.045691 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68061d1b-1c21-42dd-bc34-e320208f2caa admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:05:59.058107 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1213/2441] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:05:59 2022] PUT /identity/v3/projects/7f9817356e674c9c8bae5825a3ca9ce2/users/137fb55cf6e24c03a740e88e24694704/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:05:59.062234 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917bc816-5836-4859-a7a3-53de231604d4 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:05:59.073311 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917bc816-5836-4859-a7a3-53de231604d4 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:05:59.127976 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24235c77-f00b-43db-a5dc-f16e31bf72f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:05:59.128667 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24235c77-f00b-43db-a5dc-f16e31bf72f9 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:05:59.128959 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24235c77-f00b-43db-a5dc-f16e31bf72f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:05:59.129234 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24235c77-f00b-43db-a5dc-f16e31bf72f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:05:59.129946 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24235c77-f00b-43db-a5dc-f16e31bf72f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:05:59.132052 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24235c77-f00b-43db-a5dc-f16e31bf72f9 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:05:59.142090 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1229/2442] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:05:59.146364 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68061d1b-1c21-42dd-bc34-e320208f2caa admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:05:59.157642 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68061d1b-1c21-42dd-bc34-e320208f2caa admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:05:59.162220 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051704c9-2a8a-483b-9830-4842a1a9673a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:05:59.163738 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051704c9-2a8a-483b-9830-4842a1a9673a admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:05:59.164152 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051704c9-2a8a-483b-9830-4842a1a9673a admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:05:59.164389 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051704c9-2a8a-483b-9830-4842a1a9673a admin admin] PATH_INFO: `/v3/projects/7f9817356e674c9c8bae5825a3ca9ce2/users/137fb55cf6e24c03a740e88e24694704/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:05:59.166475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051704c9-2a8a-483b-9830-4842a1a9673a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f9817356e674c9c8bae5825a3ca9ce2, user_id=137fb55cf6e24c03a740e88e24694704, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:05:59.169158 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051704c9-2a8a-483b-9830-4842a1a9673a admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:05:59.180128 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1214/2443] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:05:59 2022] PUT /identity/v3/projects/7f9817356e674c9c8bae5825a3ca9ce2/users/137fb55cf6e24c03a740e88e24694704/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:05:59.190263 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2431f8-8d17-4ee8-a294-1ca44fcf2092 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:05:59.190561 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2431f8-8d17-4ee8-a294-1ca44fcf2092 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:05:59.190851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2431f8-8d17-4ee8-a294-1ca44fcf2092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:05:59.218710 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-6d2431f8-8d17-4ee8-a294-1ca44fcf2092 None None] MFA Rules not processed for user `137fb55cf6e24c03a740e88e24694704`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:05:59.219957 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-6d2431f8-8d17-4ee8-a294-1ca44fcf2092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:05:59.289202 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1230/2444] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:05:59 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 15:05:59.299822 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051704c9-2a8a-483b-9830-4842a1a9673a admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:05:59.301603 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-051704c9-2a8a-483b-9830-4842a1a9673a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:05:59.358552 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051704c9-2a8a-483b-9830-4842a1a9673a admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:05:59.362194 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84ba49-2e60-4f34-9d28-b21adcea6397 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:05:59.362835 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84ba49-2e60-4f34-9d28-b21adcea6397 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:05:59.363133 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84ba49-2e60-4f34-9d28-b21adcea6397 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:05:59.363402 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84ba49-2e60-4f34-9d28-b21adcea6397 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:05:59.372620 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e84ba49-2e60-4f34-9d28-b21adcea6397 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:05:59.374061 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e84ba49-2e60-4f34-9d28-b21adcea6397 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:05:59.383417 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1215/2445] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:05:59 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:05:59.521081 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2431f8-8d17-4ee8-a294-1ca44fcf2092 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:05:59.582742 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2431f8-8d17-4ee8-a294-1ca44fcf2092 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:05:59.587532 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a57c14f-96fd-482e-8b44-dee951d5c2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:05:59.588396 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a57c14f-96fd-482e-8b44-dee951d5c2f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:05:59.588714 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a57c14f-96fd-482e-8b44-dee951d5c2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:05:59.588989 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a57c14f-96fd-482e-8b44-dee951d5c2f3 admin admin] PATH_INFO: `/v3/users/bbc1dc9335824dcba4c1d2c25dfb8684` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:05:59.590193 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a57c14f-96fd-482e-8b44-dee951d5c2f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbc1dc9335824dcba4c1d2c25dfb8684)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:05:59.592103 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a57c14f-96fd-482e-8b44-dee951d5c2f3 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:05:59.639138 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9a57c14f-96fd-482e-8b44-dee951d5c2f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbc1dc9335824dcba4c1d2c25dfb8684'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:05:59.646538 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9a57c14f-96fd-482e-8b44-dee951d5c2f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbc1dc9335824dcba4c1d2c25dfb8684'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:05:59.653788 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9a57c14f-96fd-482e-8b44-dee951d5c2f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbc1dc9335824dcba4c1d2c25dfb8684'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:05:59.654442 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9a57c14f-96fd-482e-8b44-dee951d5c2f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbc1dc9335824dcba4c1d2c25dfb8684'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:05:59.655045 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9a57c14f-96fd-482e-8b44-dee951d5c2f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbc1dc9335824dcba4c1d2c25dfb8684'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:05:59.672886 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1231/2446] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:05:59 2022] DELETE /identity/v3/users/bbc1dc9335824dcba4c1d2c25dfb8684 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:05:59.677396 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a406a0a-b6f2-475b-9b09-87e798e8138e None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:05:59.677685 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a406a0a-b6f2-475b-9b09-87e798e8138e None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:05:59.677964 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a406a0a-b6f2-475b-9b09-87e798e8138e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:05:59.696803 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-5a406a0a-b6f2-475b-9b09-87e798e8138e None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:05:59.698009 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-5a406a0a-b6f2-475b-9b09-87e798e8138e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:05:59.755189 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1216/2447] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:05:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:05:59.765190 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a57c14f-96fd-482e-8b44-dee951d5c2f3 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:05:59.766632 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-9a57c14f-96fd-482e-8b44-dee951d5c2f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:05:59.824559 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a57c14f-96fd-482e-8b44-dee951d5c2f3 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:05:59.828171 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed474e9-3bf8-40eb-a175-054b6ab35c0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:05:59.828838 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed474e9-3bf8-40eb-a175-054b6ab35c0b service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:05:59.829118 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed474e9-3bf8-40eb-a175-054b6ab35c0b service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:05:59.829391 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed474e9-3bf8-40eb-a175-054b6ab35c0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:05:59.838550 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed474e9-3bf8-40eb-a175-054b6ab35c0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:05:59.839894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed474e9-3bf8-40eb-a175-054b6ab35c0b service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:05:59.849303 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1232/2448] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:05:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:06:00.112295 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a406a0a-b6f2-475b-9b09-87e798e8138e None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:00.113841 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-5a406a0a-b6f2-475b-9b09-87e798e8138e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:00.126386 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a406a0a-b6f2-475b-9b09-87e798e8138e None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:00.131359 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d4dd8b-831f-42e1-bf7d-3ff829f5ce6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:00.132991 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d4dd8b-831f-42e1-bf7d-3ff829f5ce6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:00.133205 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d4dd8b-831f-42e1-bf7d-3ff829f5ce6e admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:00.133485 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d4dd8b-831f-42e1-bf7d-3ff829f5ce6e admin admin] PATH_INFO: `/v3/projects/837aa9d1761e4608bc8300c52c1db311` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:00.135552 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d4dd8b-831f-42e1-bf7d-3ff829f5ce6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=837aa9d1761e4608bc8300c52c1db311)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:00.137362 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d4dd8b-831f-42e1-bf7d-3ff829f5ce6e admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:00.185356 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d3d4dd8b-831f-42e1-bf7d-3ff829f5ce6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '837aa9d1761e4608bc8300c52c1db311', 'request_id': 'req-d3d4dd8b-831f-42e1-bf7d-3ff829f5ce6e'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:00.190540 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d3d4dd8b-831f-42e1-bf7d-3ff829f5ce6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '837aa9d1761e4608bc8300c52c1db311', 'request_id': 'req-d3d4dd8b-831f-42e1-bf7d-3ff829f5ce6e'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:00.198302 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d3d4dd8b-831f-42e1-bf7d-3ff829f5ce6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '837aa9d1761e4608bc8300c52c1db311', 'request_id': 'req-d3d4dd8b-831f-42e1-bf7d-3ff829f5ce6e'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:00.209307 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d3d4dd8b-831f-42e1-bf7d-3ff829f5ce6e admin admin] The token cache is being invalidate because project 837aa9d1761e4608bc8300c52c1db311 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:06:00.210172 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d3d4dd8b-831f-42e1-bf7d-3ff829f5ce6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:00.211395 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d3d4dd8b-831f-42e1-bf7d-3ff829f5ce6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:00.212996 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1217/2449] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:06:00 2022] DELETE /identity/v3/projects/837aa9d1761e4608bc8300c52c1db311 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:06:00.217022 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed474e9-3bf8-40eb-a175-054b6ab35c0b service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:00.227319 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed474e9-3bf8-40eb-a175-054b6ab35c0b service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:00.281156 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f5dd5-e27a-4eae-a3d1-b58ebb222aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:00.282026 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f5dd5-e27a-4eae-a3d1-b58ebb222aab admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:00.282305 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f5dd5-e27a-4eae-a3d1-b58ebb222aab admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:00.282581 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f5dd5-e27a-4eae-a3d1-b58ebb222aab admin admin] PATH_INFO: `/v3/users/137fb55cf6e24c03a740e88e24694704` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:00.283775 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f5dd5-e27a-4eae-a3d1-b58ebb222aab admin admin] RBAC: Authorizing `identity:delete_user(user_id=137fb55cf6e24c03a740e88e24694704)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:00.285743 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f5dd5-e27a-4eae-a3d1-b58ebb222aab admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:00.331335 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ba4f5dd5-e27a-4eae-a3d1-b58ebb222aab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '137fb55cf6e24c03a740e88e24694704'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:00.338608 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ba4f5dd5-e27a-4eae-a3d1-b58ebb222aab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '137fb55cf6e24c03a740e88e24694704'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:00.345667 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ba4f5dd5-e27a-4eae-a3d1-b58ebb222aab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '137fb55cf6e24c03a740e88e24694704'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:00.346331 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ba4f5dd5-e27a-4eae-a3d1-b58ebb222aab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '137fb55cf6e24c03a740e88e24694704'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:00.346936 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ba4f5dd5-e27a-4eae-a3d1-b58ebb222aab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '137fb55cf6e24c03a740e88e24694704'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:00.365698 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1233/2450] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:06:00 2022] DELETE /identity/v3/users/137fb55cf6e24c03a740e88e24694704 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:06:00.635005 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d4dd8b-831f-42e1-bf7d-3ff829f5ce6e admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:00.636512 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d3d4dd8b-831f-42e1-bf7d-3ff829f5ce6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:00.650828 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d4dd8b-831f-42e1-bf7d-3ff829f5ce6e admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:00.712392 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db9a069-485a-45bf-80bc-6a44d47ddd89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:00.713292 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db9a069-485a-45bf-80bc-6a44d47ddd89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:00.713572 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db9a069-485a-45bf-80bc-6a44d47ddd89 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:00.713845 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db9a069-485a-45bf-80bc-6a44d47ddd89 admin admin] PATH_INFO: `/v3/projects/7f9817356e674c9c8bae5825a3ca9ce2` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:00.715031 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db9a069-485a-45bf-80bc-6a44d47ddd89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f9817356e674c9c8bae5825a3ca9ce2)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:00.716966 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db9a069-485a-45bf-80bc-6a44d47ddd89 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:00.762279 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-5db9a069-485a-45bf-80bc-6a44d47ddd89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f9817356e674c9c8bae5825a3ca9ce2', 'request_id': 'req-5db9a069-485a-45bf-80bc-6a44d47ddd89'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:00.767702 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-5db9a069-485a-45bf-80bc-6a44d47ddd89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f9817356e674c9c8bae5825a3ca9ce2', 'request_id': 'req-5db9a069-485a-45bf-80bc-6a44d47ddd89'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:00.775888 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-5db9a069-485a-45bf-80bc-6a44d47ddd89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f9817356e674c9c8bae5825a3ca9ce2', 'request_id': 'req-5db9a069-485a-45bf-80bc-6a44d47ddd89'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:00.787035 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-5db9a069-485a-45bf-80bc-6a44d47ddd89 admin admin] The token cache is being invalidate because project 7f9817356e674c9c8bae5825a3ca9ce2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:06:00.787321 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-5db9a069-485a-45bf-80bc-6a44d47ddd89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:00.788056 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-5db9a069-485a-45bf-80bc-6a44d47ddd89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:00.789562 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1218/2451] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:06:00 2022] DELETE /identity/v3/projects/7f9817356e674c9c8bae5825a3ca9ce2 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:06:00.794651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23587a-1dd7-43be-9e50-bcc02ef12a39 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:00.794968 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23587a-1dd7-43be-9e50-bcc02ef12a39 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:00.795270 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23587a-1dd7-43be-9e50-bcc02ef12a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:00.813840 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-da23587a-1dd7-43be-9e50-bcc02ef12a39 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:06:00.815033 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-da23587a-1dd7-43be-9e50-bcc02ef12a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:00.872787 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1234/2452] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:06:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:06:00.880626 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6c25a-6ca9-4059-958e-e56e70e24eb4 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:00.880871 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6c25a-6ca9-4059-958e-e56e70e24eb4 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:00.881172 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6c25a-6ca9-4059-958e-e56e70e24eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:00.900721 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-a3e6c25a-6ca9-4059-958e-e56e70e24eb4 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:06:00.902164 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-a3e6c25a-6ca9-4059-958e-e56e70e24eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:00.909886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1219/2453] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:06:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:06:00.914624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23587a-1dd7-43be-9e50-bcc02ef12a39 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:00.924152 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23587a-1dd7-43be-9e50-bcc02ef12a39 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:00.928838 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc979b-7b7c-4d05-b2ee-3c72297bb8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:00.929566 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc979b-7b7c-4d05-b2ee-3c72297bb8d4 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:00.929857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc979b-7b7c-4d05-b2ee-3c72297bb8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:00.930220 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc979b-7b7c-4d05-b2ee-3c72297bb8d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:00.930850 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc979b-7b7c-4d05-b2ee-3c72297bb8d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 15:06:00.931160 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc979b-7b7c-4d05-b2ee-3c72297bb8d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:00.932806 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc979b-7b7c-4d05-b2ee-3c72297bb8d4 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:00.944326 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1235/2454] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 15:06:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 15:06:00.948793 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e6c25a-6ca9-4059-958e-e56e70e24eb4 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:00.957974 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e6c25a-6ca9-4059-958e-e56e70e24eb4 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:00.963304 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ad2ffd-ff1b-438b-8e69-bcc36228d969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:00.963952 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad2ffd-ff1b-438b-8e69-bcc36228d969 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:00.964237 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad2ffd-ff1b-438b-8e69-bcc36228d969 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:00.964518 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad2ffd-ff1b-438b-8e69-bcc36228d969 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:00.965238 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ad2ffd-ff1b-438b-8e69-bcc36228d969 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:00.967198 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ad2ffd-ff1b-438b-8e69-bcc36228d969 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:00.992286 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1220/2455] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:06:00 2022] POST /identity/v3/projects => generated 379 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:06:00.996427 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc979b-7b7c-4d05-b2ee-3c72297bb8d4 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:01.006224 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc979b-7b7c-4d05-b2ee-3c72297bb8d4 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:01.061542 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac527e76-2a48-412d-a070-4aa90230ab3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:01.062209 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac527e76-2a48-412d-a070-4aa90230ab3c admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:01.062491 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac527e76-2a48-412d-a070-4aa90230ab3c admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:01.062767 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac527e76-2a48-412d-a070-4aa90230ab3c admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:01.063481 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac527e76-2a48-412d-a070-4aa90230ab3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:01.065413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac527e76-2a48-412d-a070-4aa90230ab3c admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:01.085365 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1236/2456] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:06:00 2022] POST /identity/v3/users => generated 290 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:06:01.089679 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ad2ffd-ff1b-438b-8e69-bcc36228d969 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:01.099858 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ad2ffd-ff1b-438b-8e69-bcc36228d969 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:01.105093 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614981f2-f3b8-4592-b035-34509be55463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:01.105741 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614981f2-f3b8-4592-b035-34509be55463 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:01.106024 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614981f2-f3b8-4592-b035-34509be55463 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:01.106297 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614981f2-f3b8-4592-b035-34509be55463 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:01.107321 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614981f2-f3b8-4592-b035-34509be55463 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:01.109254 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614981f2-f3b8-4592-b035-34509be55463 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:01.120174 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1221/2457] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:06:01.124155 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac527e76-2a48-412d-a070-4aa90230ab3c admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:01.134840 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac527e76-2a48-412d-a070-4aa90230ab3c admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:01.139448 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ac63b-8986-4f94-88ad-628dab886ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:01.140105 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ac63b-8986-4f94-88ad-628dab886ee2 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:01.140394 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ac63b-8986-4f94-88ad-628dab886ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:01.140702 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ac63b-8986-4f94-88ad-628dab886ee2 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:01.141423 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568ac63b-8986-4f94-88ad-628dab886ee2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:01.143048 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568ac63b-8986-4f94-88ad-628dab886ee2 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:01.153347 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1237/2458] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:06:01.157552 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614981f2-f3b8-4592-b035-34509be55463 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:01.168461 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614981f2-f3b8-4592-b035-34509be55463 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:01.173238 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e48dfe-b59c-41ca-8a9c-8617a4e61478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:01.174135 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e48dfe-b59c-41ca-8a9c-8617a4e61478 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:01.174416 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e48dfe-b59c-41ca-8a9c-8617a4e61478 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:01.174859 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e48dfe-b59c-41ca-8a9c-8617a4e61478 admin admin] PATH_INFO: `/v3/projects/45f7e1cf4a9741a59bf8152f7afdf877/users/9109c7182d424fdeab41dcad32c0bcac/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:01.191371 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e48dfe-b59c-41ca-8a9c-8617a4e61478 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45f7e1cf4a9741a59bf8152f7afdf877, user_id=9109c7182d424fdeab41dcad32c0bcac, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:01.194570 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e48dfe-b59c-41ca-8a9c-8617a4e61478 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:01.211788 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1222/2459] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:06:01 2022] PUT /identity/v3/projects/45f7e1cf4a9741a59bf8152f7afdf877/users/9109c7182d424fdeab41dcad32c0bcac/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:06:01.223009 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac642a04-3a7d-49b1-964d-5cd3698a1911 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:01.223538 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac642a04-3a7d-49b1-964d-5cd3698a1911 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:01.223635 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac642a04-3a7d-49b1-964d-5cd3698a1911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:01.252024 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-ac642a04-3a7d-49b1-964d-5cd3698a1911 None None] MFA Rules not processed for user `9109c7182d424fdeab41dcad32c0bcac`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:06:01.253330 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-ac642a04-3a7d-49b1-964d-5cd3698a1911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:01.321285 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1238/2460] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:06:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:06:01.325846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e48dfe-b59c-41ca-8a9c-8617a4e61478 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:01.388122 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e48dfe-b59c-41ca-8a9c-8617a4e61478 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:01.392952 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6242e01-b36c-4ae2-b73b-ad715bfd86ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:01.393615 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6242e01-b36c-4ae2-b73b-ad715bfd86ab admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:01.393902 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6242e01-b36c-4ae2-b73b-ad715bfd86ab admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:01.394183 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6242e01-b36c-4ae2-b73b-ad715bfd86ab admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:01.394912 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6242e01-b36c-4ae2-b73b-ad715bfd86ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:01.396943 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6242e01-b36c-4ae2-b73b-ad715bfd86ab admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:01.422607 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1223/2461] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:06:01 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 15:06:01.426797 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac642a04-3a7d-49b1-964d-5cd3698a1911 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:01.438271 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac642a04-3a7d-49b1-964d-5cd3698a1911 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:01.495556 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc340c-f88f-4348-b7bd-47e88e643fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:01.496238 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc340c-f88f-4348-b7bd-47e88e643fd3 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:01.496526 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc340c-f88f-4348-b7bd-47e88e643fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:01.496867 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc340c-f88f-4348-b7bd-47e88e643fd3 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:01.497587 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bc340c-f88f-4348-b7bd-47e88e643fd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:01.499492 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bc340c-f88f-4348-b7bd-47e88e643fd3 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:01.520209 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1239/2462] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:06:01 2022] POST /identity/v3/users => generated 298 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:06:01.525084 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6242e01-b36c-4ae2-b73b-ad715bfd86ab admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:01.535494 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6242e01-b36c-4ae2-b73b-ad715bfd86ab admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:01.540223 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32190d47-deaf-412e-9cb4-bfa300089e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:01.540906 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32190d47-deaf-412e-9cb4-bfa300089e5b admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:01.541191 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32190d47-deaf-412e-9cb4-bfa300089e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:01.541472 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32190d47-deaf-412e-9cb4-bfa300089e5b admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:01.542188 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32190d47-deaf-412e-9cb4-bfa300089e5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:01.543804 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32190d47-deaf-412e-9cb4-bfa300089e5b admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:01.554862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1224/2463] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:06:01.558952 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc340c-f88f-4348-b7bd-47e88e643fd3 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:01.568595 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc340c-f88f-4348-b7bd-47e88e643fd3 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:01.573946 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37099778-49e9-4fef-88d4-0f56a6825283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:01.574816 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37099778-49e9-4fef-88d4-0f56a6825283 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:01.575098 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37099778-49e9-4fef-88d4-0f56a6825283 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:01.575370 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37099778-49e9-4fef-88d4-0f56a6825283 admin admin] PATH_INFO: `/v3/projects/7d3bd11c688c4143ae3f9f8ea0bd9c4a/users/934ba49d80594a3698f0c9a9cbb02a51/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:01.590125 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37099778-49e9-4fef-88d4-0f56a6825283 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d3bd11c688c4143ae3f9f8ea0bd9c4a, user_id=934ba49d80594a3698f0c9a9cbb02a51, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:01.592782 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37099778-49e9-4fef-88d4-0f56a6825283 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:01.605057 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1240/2464] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:06:01 2022] PUT /identity/v3/projects/7d3bd11c688c4143ae3f9f8ea0bd9c4a/users/934ba49d80594a3698f0c9a9cbb02a51/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:06:01.608959 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32190d47-deaf-412e-9cb4-bfa300089e5b admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:01.619528 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32190d47-deaf-412e-9cb4-bfa300089e5b admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:01.675857 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b334fb6-1310-4114-8741-77fe3465cd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:01.676442 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b334fb6-1310-4114-8741-77fe3465cd0e admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:01.676751 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b334fb6-1310-4114-8741-77fe3465cd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:01.677025 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b334fb6-1310-4114-8741-77fe3465cd0e admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:01.677739 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b334fb6-1310-4114-8741-77fe3465cd0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:01.679892 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b334fb6-1310-4114-8741-77fe3465cd0e admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:01.689466 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1225/2465] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 06 15:06:01.693558 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37099778-49e9-4fef-88d4-0f56a6825283 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:01.704566 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37099778-49e9-4fef-88d4-0f56a6825283 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:01.709418 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b54240-129e-4642-938f-852dea26d02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:01.710515 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b54240-129e-4642-938f-852dea26d02a admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:01.710802 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b54240-129e-4642-938f-852dea26d02a admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:01.711084 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b54240-129e-4642-938f-852dea26d02a admin admin] PATH_INFO: `/v3/projects/7d3bd11c688c4143ae3f9f8ea0bd9c4a/users/934ba49d80594a3698f0c9a9cbb02a51/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:01.713106 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b54240-129e-4642-938f-852dea26d02a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d3bd11c688c4143ae3f9f8ea0bd9c4a, user_id=934ba49d80594a3698f0c9a9cbb02a51, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:01.715737 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b54240-129e-4642-938f-852dea26d02a admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:01.726059 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1241/2466] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:06:01 2022] PUT /identity/v3/projects/7d3bd11c688c4143ae3f9f8ea0bd9c4a/users/934ba49d80594a3698f0c9a9cbb02a51/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:06:01.736148 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698212b6-26f9-4392-ae7d-e3ff67350209 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:01.736442 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698212b6-26f9-4392-ae7d-e3ff67350209 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:01.736770 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698212b6-26f9-4392-ae7d-e3ff67350209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:01.763302 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-698212b6-26f9-4392-ae7d-e3ff67350209 None None] MFA Rules not processed for user `934ba49d80594a3698f0c9a9cbb02a51`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:06:01.764515 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-698212b6-26f9-4392-ae7d-e3ff67350209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:01.831860 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1226/2467] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:06:01 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:06:01.843142 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b54240-129e-4642-938f-852dea26d02a admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:01.844879 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-96b54240-129e-4642-938f-852dea26d02a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:01.902303 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b54240-129e-4642-938f-852dea26d02a admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:01.906093 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133a2090-fba5-4871-ab41-c8bf27a7180c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:01.906760 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a2090-fba5-4871-ab41-c8bf27a7180c service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:01.907045 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a2090-fba5-4871-ab41-c8bf27a7180c service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:01.907323 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a2090-fba5-4871-ab41-c8bf27a7180c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:01.917717 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133a2090-fba5-4871-ab41-c8bf27a7180c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:01.919147 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133a2090-fba5-4871-ab41-c8bf27a7180c service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:01.928718 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1242/2468] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:06:01 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:06:02.031828 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698212b6-26f9-4392-ae7d-e3ff67350209 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:02.041374 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698212b6-26f9-4392-ae7d-e3ff67350209 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:02.045186 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d6055-4628-4818-b74d-d71cb723a974 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:02.045863 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d6055-4628-4818-b74d-d71cb723a974 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:02.046148 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d6055-4628-4818-b74d-d71cb723a974 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:02.046430 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d6055-4628-4818-b74d-d71cb723a974 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:02.101067 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1d6055-4628-4818-b74d-d71cb723a974 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:02.102510 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1d6055-4628-4818-b74d-d71cb723a974 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:02.111478 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1227/2469] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:06:02 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:06:04.690077 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133a2090-fba5-4871-ab41-c8bf27a7180c service cinder] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:04.760129 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133a2090-fba5-4871-ab41-c8bf27a7180c service cinder] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:04.767677 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9478bd74-5576-4a49-a698-081ebabe1913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:04.769137 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9478bd74-5576-4a49-a698-081ebabe1913 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:04.769653 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9478bd74-5576-4a49-a698-081ebabe1913 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:04.770168 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9478bd74-5576-4a49-a698-081ebabe1913 admin admin] PATH_INFO: `/v3/users/9109c7182d424fdeab41dcad32c0bcac` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:04.772951 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9478bd74-5576-4a49-a698-081ebabe1913 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9109c7182d424fdeab41dcad32c0bcac)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:04.776558 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9478bd74-5576-4a49-a698-081ebabe1913 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:04.839184 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9478bd74-5576-4a49-a698-081ebabe1913 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9109c7182d424fdeab41dcad32c0bcac'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:04.849525 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9478bd74-5576-4a49-a698-081ebabe1913 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9109c7182d424fdeab41dcad32c0bcac'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:04.857758 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9478bd74-5576-4a49-a698-081ebabe1913 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9109c7182d424fdeab41dcad32c0bcac'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:04.858485 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9478bd74-5576-4a49-a698-081ebabe1913 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9109c7182d424fdeab41dcad32c0bcac'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:04.859216 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9478bd74-5576-4a49-a698-081ebabe1913 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9109c7182d424fdeab41dcad32c0bcac'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:04.878315 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1243/2470] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:06:04 2022] DELETE /identity/v3/users/9109c7182d424fdeab41dcad32c0bcac => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:06:04.882710 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e3b16-ba97-442f-a7ef-c972e78e4f37 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:04.883249 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e3b16-ba97-442f-a7ef-c972e78e4f37 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:04.883642 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e3b16-ba97-442f-a7ef-c972e78e4f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:04.906281 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-5e4e3b16-ba97-442f-a7ef-c972e78e4f37 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:06:04.908054 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-5e4e3b16-ba97-442f-a7ef-c972e78e4f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:04.974443 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1228/2471] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:06:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:06:04.985245 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9478bd74-5576-4a49-a698-081ebabe1913 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:04.986805 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-9478bd74-5576-4a49-a698-081ebabe1913 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:05.067633 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9478bd74-5576-4a49-a698-081ebabe1913 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:05.071085 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3aa2b8-512c-4e0f-8f13-e52da7727c02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:05.071743 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3aa2b8-512c-4e0f-8f13-e52da7727c02 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:05.072026 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3aa2b8-512c-4e0f-8f13-e52da7727c02 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:05.072330 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3aa2b8-512c-4e0f-8f13-e52da7727c02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:05.082336 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3aa2b8-512c-4e0f-8f13-e52da7727c02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:05.085587 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3aa2b8-512c-4e0f-8f13-e52da7727c02 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:05.097839 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1244/2472] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:06:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:06:05.368982 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4e3b16-ba97-442f-a7ef-c972e78e4f37 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:05.370726 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-5e4e3b16-ba97-442f-a7ef-c972e78e4f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:05.393623 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4e3b16-ba97-442f-a7ef-c972e78e4f37 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:05.400266 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef295059-7805-4251-bb08-5c81b30d228b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:05.401469 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef295059-7805-4251-bb08-5c81b30d228b admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:05.401469 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef295059-7805-4251-bb08-5c81b30d228b admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:05.402081 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef295059-7805-4251-bb08-5c81b30d228b admin admin] PATH_INFO: `/v3/projects/45f7e1cf4a9741a59bf8152f7afdf877` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:05.403338 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef295059-7805-4251-bb08-5c81b30d228b admin admin] RBAC: Authorizing `identity:delete_project(project_id=45f7e1cf4a9741a59bf8152f7afdf877)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:05.405326 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef295059-7805-4251-bb08-5c81b30d228b admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:05.453729 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-ef295059-7805-4251-bb08-5c81b30d228b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45f7e1cf4a9741a59bf8152f7afdf877', 'request_id': 'req-ef295059-7805-4251-bb08-5c81b30d228b'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:05.458916 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-ef295059-7805-4251-bb08-5c81b30d228b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45f7e1cf4a9741a59bf8152f7afdf877', 'request_id': 'req-ef295059-7805-4251-bb08-5c81b30d228b'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:05.467298 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-ef295059-7805-4251-bb08-5c81b30d228b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45f7e1cf4a9741a59bf8152f7afdf877', 'request_id': 'req-ef295059-7805-4251-bb08-5c81b30d228b'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:05.479300 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-ef295059-7805-4251-bb08-5c81b30d228b admin admin] The token cache is being invalidate because project 45f7e1cf4a9741a59bf8152f7afdf877 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:06:05.479588 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-ef295059-7805-4251-bb08-5c81b30d228b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:05.480232 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-ef295059-7805-4251-bb08-5c81b30d228b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:05.481833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1229/2473] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:06:05 2022] DELETE /identity/v3/projects/45f7e1cf4a9741a59bf8152f7afdf877 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 15:06:05.485719 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3aa2b8-512c-4e0f-8f13-e52da7727c02 service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:05.496372 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3aa2b8-512c-4e0f-8f13-e52da7727c02 service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:05.559990 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b152c139-80da-4611-9a0b-59878f3d20c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:05.560909 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b152c139-80da-4611-9a0b-59878f3d20c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:05.561192 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b152c139-80da-4611-9a0b-59878f3d20c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:05.561467 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b152c139-80da-4611-9a0b-59878f3d20c5 admin admin] PATH_INFO: `/v3/users/934ba49d80594a3698f0c9a9cbb02a51` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:05.562666 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b152c139-80da-4611-9a0b-59878f3d20c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=934ba49d80594a3698f0c9a9cbb02a51)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:05.564628 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b152c139-80da-4611-9a0b-59878f3d20c5 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:05.611969 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b152c139-80da-4611-9a0b-59878f3d20c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '934ba49d80594a3698f0c9a9cbb02a51'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:05.619364 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b152c139-80da-4611-9a0b-59878f3d20c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '934ba49d80594a3698f0c9a9cbb02a51'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:05.626594 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b152c139-80da-4611-9a0b-59878f3d20c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '934ba49d80594a3698f0c9a9cbb02a51'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:05.627320 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b152c139-80da-4611-9a0b-59878f3d20c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '934ba49d80594a3698f0c9a9cbb02a51'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:05.627937 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b152c139-80da-4611-9a0b-59878f3d20c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '934ba49d80594a3698f0c9a9cbb02a51'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:05.646114 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1245/2474] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:06:05 2022] DELETE /identity/v3/users/934ba49d80594a3698f0c9a9cbb02a51 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:06:05.964923 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef295059-7805-4251-bb08-5c81b30d228b admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:05.967163 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ef295059-7805-4251-bb08-5c81b30d228b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:05.983210 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef295059-7805-4251-bb08-5c81b30d228b admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:06.045798 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01445c19-ec06-44a1-a6a8-ad92e592ec4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:06.046802 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01445c19-ec06-44a1-a6a8-ad92e592ec4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:06.046953 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01445c19-ec06-44a1-a6a8-ad92e592ec4a admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:06.047232 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01445c19-ec06-44a1-a6a8-ad92e592ec4a admin admin] PATH_INFO: `/v3/projects/7d3bd11c688c4143ae3f9f8ea0bd9c4a` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:06.048518 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01445c19-ec06-44a1-a6a8-ad92e592ec4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d3bd11c688c4143ae3f9f8ea0bd9c4a)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:06.050487 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01445c19-ec06-44a1-a6a8-ad92e592ec4a admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:06.107225 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-01445c19-ec06-44a1-a6a8-ad92e592ec4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d3bd11c688c4143ae3f9f8ea0bd9c4a', 'request_id': 'req-01445c19-ec06-44a1-a6a8-ad92e592ec4a'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:06.115303 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-01445c19-ec06-44a1-a6a8-ad92e592ec4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d3bd11c688c4143ae3f9f8ea0bd9c4a', 'request_id': 'req-01445c19-ec06-44a1-a6a8-ad92e592ec4a'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:06.123782 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-01445c19-ec06-44a1-a6a8-ad92e592ec4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d3bd11c688c4143ae3f9f8ea0bd9c4a', 'request_id': 'req-01445c19-ec06-44a1-a6a8-ad92e592ec4a'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:06.136465 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-01445c19-ec06-44a1-a6a8-ad92e592ec4a admin admin] The token cache is being invalidate because project 7d3bd11c688c4143ae3f9f8ea0bd9c4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:06:06.136874 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-01445c19-ec06-44a1-a6a8-ad92e592ec4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:06.137635 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-01445c19-ec06-44a1-a6a8-ad92e592ec4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:06.139487 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1230/2475] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:06:05 2022] DELETE /identity/v3/projects/7d3bd11c688c4143ae3f9f8ea0bd9c4a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:06:06.144143 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f56d1-b586-4be3-aa95-9324b21a0bb2 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:06.144714 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f56d1-b586-4be3-aa95-9324b21a0bb2 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:06.144823 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f56d1-b586-4be3-aa95-9324b21a0bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:06.167582 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-910f56d1-b586-4be3-aa95-9324b21a0bb2 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:06:06.168840 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-910f56d1-b586-4be3-aa95-9324b21a0bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:06.234485 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1246/2476] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:06:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 15:06:06.243150 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54cee7c-d1d9-4318-8d61-760d0a9c5e73 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:06.243435 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54cee7c-d1d9-4318-8d61-760d0a9c5e73 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:06.243708 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54cee7c-d1d9-4318-8d61-760d0a9c5e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:06.262124 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-d54cee7c-d1d9-4318-8d61-760d0a9c5e73 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:06:06.263519 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d54cee7c-d1d9-4318-8d61-760d0a9c5e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:06.271325 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1231/2477] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:06:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 15:06:06.276202 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910f56d1-b586-4be3-aa95-9324b21a0bb2 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:06.286061 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910f56d1-b586-4be3-aa95-9324b21a0bb2 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:06.290904 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e0afc-9597-40ee-bd7f-59b11322c650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:06.291569 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e0afc-9597-40ee-bd7f-59b11322c650 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:06.291851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e0afc-9597-40ee-bd7f-59b11322c650 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:06.292117 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e0afc-9597-40ee-bd7f-59b11322c650 admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:06.292874 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4e0afc-9597-40ee-bd7f-59b11322c650 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 15:06:06.293178 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4e0afc-9597-40ee-bd7f-59b11322c650 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:06.294825 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4e0afc-9597-40ee-bd7f-59b11322c650 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:06.307082 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1247/2478] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 15:06:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 15:06:06.311412 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54cee7c-d1d9-4318-8d61-760d0a9c5e73 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:06.321085 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54cee7c-d1d9-4318-8d61-760d0a9c5e73 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:06.325800 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e8cc6-0d0c-45d5-8240-dce9cc1c96a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:06.326453 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e8cc6-0d0c-45d5-8240-dce9cc1c96a9 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:06.326732 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e8cc6-0d0c-45d5-8240-dce9cc1c96a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:06.327011 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e8cc6-0d0c-45d5-8240-dce9cc1c96a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:06.327691 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54e8cc6-0d0c-45d5-8240-dce9cc1c96a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:06.329635 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54e8cc6-0d0c-45d5-8240-dce9cc1c96a9 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:06.360313 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1232/2479] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:06:06 2022] POST /identity/v3/projects => generated 409 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:06:06.361492 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e0afc-9597-40ee-bd7f-59b11322c650 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:06.371450 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e0afc-9597-40ee-bd7f-59b11322c650 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:06.429870 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f0bd4-5119-4119-986c-0b0f56d6c780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:06.430571 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f0bd4-5119-4119-986c-0b0f56d6c780 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:06.430887 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f0bd4-5119-4119-986c-0b0f56d6c780 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:06.431169 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f0bd4-5119-4119-986c-0b0f56d6c780 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:06.431852 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90f0bd4-5119-4119-986c-0b0f56d6c780 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:06.433810 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90f0bd4-5119-4119-986c-0b0f56d6c780 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:06.453916 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1248/2480] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:06:06 2022] POST /identity/v3/users => generated 305 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:06:06.458063 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e8cc6-0d0c-45d5-8240-dce9cc1c96a9 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:06.468975 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e8cc6-0d0c-45d5-8240-dce9cc1c96a9 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:06.473503 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6120b293-e9cf-4498-b020-8b13d3b3d13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:06.474166 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120b293-e9cf-4498-b020-8b13d3b3d13a admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:06.474446 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120b293-e9cf-4498-b020-8b13d3b3d13a admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:06.474725 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120b293-e9cf-4498-b020-8b13d3b3d13a admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:06.475475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6120b293-e9cf-4498-b020-8b13d3b3d13a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:06.477069 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6120b293-e9cf-4498-b020-8b13d3b3d13a admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:06.487180 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1233/2481] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:06:06.491256 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f0bd4-5119-4119-986c-0b0f56d6c780 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:06.501057 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f0bd4-5119-4119-986c-0b0f56d6c780 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:06.505625 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db99ede3-ed65-4cd2-90e8-ea5f81bb5300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:06.506824 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db99ede3-ed65-4cd2-90e8-ea5f81bb5300 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:06.507107 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db99ede3-ed65-4cd2-90e8-ea5f81bb5300 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:06.507381 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db99ede3-ed65-4cd2-90e8-ea5f81bb5300 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:06.508086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db99ede3-ed65-4cd2-90e8-ea5f81bb5300 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:06.509685 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db99ede3-ed65-4cd2-90e8-ea5f81bb5300 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:06.519383 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1249/2482] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:06:06.524154 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6120b293-e9cf-4498-b020-8b13d3b3d13a admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:06.534362 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6120b293-e9cf-4498-b020-8b13d3b3d13a admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:06.538926 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e83bd1-c495-476b-a93c-9b4f8c70b63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:06.539784 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e83bd1-c495-476b-a93c-9b4f8c70b63f admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:06.540059 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e83bd1-c495-476b-a93c-9b4f8c70b63f admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:06.540327 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e83bd1-c495-476b-a93c-9b4f8c70b63f admin admin] PATH_INFO: `/v3/projects/56709156865f4164a16dddc59eca2258/users/375e3cdb49e344ae842c474ea05e4ef2/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:06.554362 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e83bd1-c495-476b-a93c-9b4f8c70b63f admin admin] RBAC: Authorizing `identity:create_grant(project_id=56709156865f4164a16dddc59eca2258, user_id=375e3cdb49e344ae842c474ea05e4ef2, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:06.557022 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e83bd1-c495-476b-a93c-9b4f8c70b63f admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:06.570364 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1234/2483] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:06:06 2022] PUT /identity/v3/projects/56709156865f4164a16dddc59eca2258/users/375e3cdb49e344ae842c474ea05e4ef2/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:06:06.581531 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4865e-9c36-4902-b20d-6de713cd75c8 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:06.581821 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4865e-9c36-4902-b20d-6de713cd75c8 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:06.582092 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4865e-9c36-4902-b20d-6de713cd75c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:06.608713 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-0ff4865e-9c36-4902-b20d-6de713cd75c8 None None] MFA Rules not processed for user `375e3cdb49e344ae842c474ea05e4ef2`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:06:06.609904 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-0ff4865e-9c36-4902-b20d-6de713cd75c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:06.675131 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1250/2484] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:06:06 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:06:06.680140 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e83bd1-c495-476b-a93c-9b4f8c70b63f admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:06.739202 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e83bd1-c495-476b-a93c-9b4f8c70b63f admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:06.743963 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ef57b-64b4-4b7d-b7ff-47c31a7c6d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:06.744628 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ef57b-64b4-4b7d-b7ff-47c31a7c6d33 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:06.744919 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ef57b-64b4-4b7d-b7ff-47c31a7c6d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:06.745194 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ef57b-64b4-4b7d-b7ff-47c31a7c6d33 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:06.745883 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76ef57b-64b4-4b7d-b7ff-47c31a7c6d33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:06.747814 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76ef57b-64b4-4b7d-b7ff-47c31a7c6d33 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:06.774564 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1235/2485] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:06:06 2022] POST /identity/v3/projects => generated 407 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:06:06.778864 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff4865e-9c36-4902-b20d-6de713cd75c8 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:06.789362 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff4865e-9c36-4902-b20d-6de713cd75c8 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:06.847555 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ae8ba-d468-4749-b629-e5e21ab88999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:06.848232 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ae8ba-d468-4749-b629-e5e21ab88999 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:06.848515 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ae8ba-d468-4749-b629-e5e21ab88999 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:06.848822 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ae8ba-d468-4749-b629-e5e21ab88999 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:06.849516 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7ae8ba-d468-4749-b629-e5e21ab88999 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:06.852055 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7ae8ba-d468-4749-b629-e5e21ab88999 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:06.871118 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1251/2486] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:06:06 2022] POST /identity/v3/users => generated 310 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:06:06.875125 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ef57b-64b4-4b7d-b7ff-47c31a7c6d33 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:06.885439 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ef57b-64b4-4b7d-b7ff-47c31a7c6d33 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:06.890025 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef3878e-7f6e-431a-aa6e-f272100c24ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:06.890674 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3878e-7f6e-431a-aa6e-f272100c24ab admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:06.890956 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3878e-7f6e-431a-aa6e-f272100c24ab admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:06.891232 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3878e-7f6e-431a-aa6e-f272100c24ab admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:06.891942 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef3878e-7f6e-431a-aa6e-f272100c24ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:06.893579 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef3878e-7f6e-431a-aa6e-f272100c24ab admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:06.905378 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1236/2487] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:06:06.909489 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ae8ba-d468-4749-b629-e5e21ab88999 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:06.920057 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ae8ba-d468-4749-b629-e5e21ab88999 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:06.924691 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1a031c-0a25-4467-8b02-82c54ef33728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:06.925560 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a031c-0a25-4467-8b02-82c54ef33728 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:06.925837 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a031c-0a25-4467-8b02-82c54ef33728 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:06.926106 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a031c-0a25-4467-8b02-82c54ef33728 admin admin] PATH_INFO: `/v3/projects/c29e8a6fadb0452b8abe6b0c3e119d48/users/f4bbdc66a79b457c8e306d2853d2882c/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:06.941402 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1a031c-0a25-4467-8b02-82c54ef33728 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c29e8a6fadb0452b8abe6b0c3e119d48, user_id=f4bbdc66a79b457c8e306d2853d2882c, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:06.944199 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1a031c-0a25-4467-8b02-82c54ef33728 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:06.956587 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1252/2488] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:06:06 2022] PUT /identity/v3/projects/c29e8a6fadb0452b8abe6b0c3e119d48/users/f4bbdc66a79b457c8e306d2853d2882c/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:06:06.961473 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef3878e-7f6e-431a-aa6e-f272100c24ab admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:06.971349 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef3878e-7f6e-431a-aa6e-f272100c24ab admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:07.028097 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8b11bc-25d4-4137-b4f2-bb576e32bf93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:07.028728 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8b11bc-25d4-4137-b4f2-bb576e32bf93 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:07.029019 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8b11bc-25d4-4137-b4f2-bb576e32bf93 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:07.029295 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8b11bc-25d4-4137-b4f2-bb576e32bf93 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:07.030010 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8b11bc-25d4-4137-b4f2-bb576e32bf93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:07.031605 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8b11bc-25d4-4137-b4f2-bb576e32bf93 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:07.041945 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1237/2489] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:06:07.046071 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1a031c-0a25-4467-8b02-82c54ef33728 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:07.056118 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1a031c-0a25-4467-8b02-82c54ef33728 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:07.060732 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f52881-3ac8-4075-ae8d-27694263b214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:07.061609 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f52881-3ac8-4075-ae8d-27694263b214 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:07.061906 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f52881-3ac8-4075-ae8d-27694263b214 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:07.062182 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f52881-3ac8-4075-ae8d-27694263b214 admin admin] PATH_INFO: `/v3/projects/c29e8a6fadb0452b8abe6b0c3e119d48/users/f4bbdc66a79b457c8e306d2853d2882c/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:07.064122 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f52881-3ac8-4075-ae8d-27694263b214 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c29e8a6fadb0452b8abe6b0c3e119d48, user_id=f4bbdc66a79b457c8e306d2853d2882c, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:07.066777 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f52881-3ac8-4075-ae8d-27694263b214 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:07.077427 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1253/2490] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:06:07 2022] PUT /identity/v3/projects/c29e8a6fadb0452b8abe6b0c3e119d48/users/f4bbdc66a79b457c8e306d2853d2882c/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:06:07.088182 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e682cea-3210-457d-a636-c74dcffef3d2 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:07.088472 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e682cea-3210-457d-a636-c74dcffef3d2 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:07.088796 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e682cea-3210-457d-a636-c74dcffef3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:07.128248 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-1e682cea-3210-457d-a636-c74dcffef3d2 None None] MFA Rules not processed for user `f4bbdc66a79b457c8e306d2853d2882c`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:06:07.129500 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-1e682cea-3210-457d-a636-c74dcffef3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:07.201703 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1238/2491] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:06:07 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:06:07.212949 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f52881-3ac8-4075-ae8d-27694263b214 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:07.214675 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-01f52881-3ac8-4075-ae8d-27694263b214 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:07.275014 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f52881-3ac8-4075-ae8d-27694263b214 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:07.278980 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea14605-02a8-4bb3-9584-06c4092a92e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:07.279568 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea14605-02a8-4bb3-9584-06c4092a92e6 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:07.279877 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea14605-02a8-4bb3-9584-06c4092a92e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:07.280111 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea14605-02a8-4bb3-9584-06c4092a92e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:07.289901 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea14605-02a8-4bb3-9584-06c4092a92e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:07.291310 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea14605-02a8-4bb3-9584-06c4092a92e6 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:07.302071 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1254/2492] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:06:07 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:06:07.463488 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e682cea-3210-457d-a636-c74dcffef3d2 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:07.464971 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-1e682cea-3210-457d-a636-c74dcffef3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:07.477608 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e682cea-3210-457d-a636-c74dcffef3d2 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:07.481200 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba629005-1a7e-4674-b2e8-36c32c158ee8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:07.481859 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba629005-1a7e-4674-b2e8-36c32c158ee8 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:07.482143 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba629005-1a7e-4674-b2e8-36c32c158ee8 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:07.482428 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba629005-1a7e-4674-b2e8-36c32c158ee8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:07.544062 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba629005-1a7e-4674-b2e8-36c32c158ee8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:07.545560 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba629005-1a7e-4674-b2e8-36c32c158ee8 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:07.556359 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1239/2493] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:06:07 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:06:14.642625 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab3e470-b06e-4ade-8faa-91a08624b7d1 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:14.643134 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab3e470-b06e-4ade-8faa-91a08624b7d1 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:14.643244 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab3e470-b06e-4ade-8faa-91a08624b7d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:14.661073 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-dab3e470-b06e-4ade-8faa-91a08624b7d1 None None] MFA Rules not processed for user `a7dc1bc53c59423e934f7831ea022c73`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:06:14.662908 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-dab3e470-b06e-4ade-8faa-91a08624b7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:14.720954 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1255/2494] 10.222.0.23 () {58 vars in 967 bytes} [Fri May 6 15:06:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:06:14.774617 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba629005-1a7e-4674-b2e8-36c32c158ee8 service cinder] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:14.776115 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ba629005-1a7e-4674-b2e8-36c32c158ee8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:14.844890 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba629005-1a7e-4674-b2e8-36c32c158ee8 service cinder] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:14.850232 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a87ec5-64c4-4124-bff4-c74301a85978 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf9eed62c8f1409282c2af6488426a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:14.850875 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a87ec5-64c4-4124-bff4-c74301a85978 service swift] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:14.851146 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a87ec5-64c4-4124-bff4-c74301a85978 service swift] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:14.851415 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a87ec5-64c4-4124-bff4-c74301a85978 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:14.859922 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a87ec5-64c4-4124-bff4-c74301a85978 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:14.861539 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a87ec5-64c4-4124-bff4-c74301a85978 service swift] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:14.870187 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1240/2495] 10.222.0.23 () {60 vars in 1384 bytes} [Fri May 6 15:06:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 06 15:06:15.977182 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab3e470-b06e-4ade-8faa-91a08624b7d1 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:16.050726 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab3e470-b06e-4ade-8faa-91a08624b7d1 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:16.055840 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2701e-828f-4bd2-98fc-c10916d179d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:16.056775 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2701e-828f-4bd2-98fc-c10916d179d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:16.057058 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2701e-828f-4bd2-98fc-c10916d179d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:16.057331 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2701e-828f-4bd2-98fc-c10916d179d9 admin admin] PATH_INFO: `/v3/users/375e3cdb49e344ae842c474ea05e4ef2` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:16.058527 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf2701e-828f-4bd2-98fc-c10916d179d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=375e3cdb49e344ae842c474ea05e4ef2)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:16.061153 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf2701e-828f-4bd2-98fc-c10916d179d9 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:16.120671 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-0bf2701e-828f-4bd2-98fc-c10916d179d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '375e3cdb49e344ae842c474ea05e4ef2'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:16.129039 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-0bf2701e-828f-4bd2-98fc-c10916d179d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '375e3cdb49e344ae842c474ea05e4ef2'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:16.137399 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-0bf2701e-828f-4bd2-98fc-c10916d179d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '375e3cdb49e344ae842c474ea05e4ef2'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:16.138147 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-0bf2701e-828f-4bd2-98fc-c10916d179d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '375e3cdb49e344ae842c474ea05e4ef2'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:16.138761 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-0bf2701e-828f-4bd2-98fc-c10916d179d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '375e3cdb49e344ae842c474ea05e4ef2'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:16.163393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1256/2496] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:06:15 2022] DELETE /identity/v3/users/375e3cdb49e344ae842c474ea05e4ef2 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:06:16.165300 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1374023-79e6-4bac-8b86-caf60a936aa9 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:16.165762 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1374023-79e6-4bac-8b86-caf60a936aa9 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:16.165880 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1374023-79e6-4bac-8b86-caf60a936aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:16.184126 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-e1374023-79e6-4bac-8b86-caf60a936aa9 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:06:16.185508 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-e1374023-79e6-4bac-8b86-caf60a936aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:16.256507 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1241/2497] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:06:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:06:16.267707 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2701e-828f-4bd2-98fc-c10916d179d9 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:16.269306 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-0bf2701e-828f-4bd2-98fc-c10916d179d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:16.337459 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2701e-828f-4bd2-98fc-c10916d179d9 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:16.341471 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee63f0-f0db-47b6-9bb0-2d5a10f666ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:16.342204 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee63f0-f0db-47b6-9bb0-2d5a10f666ab service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:16.342487 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee63f0-f0db-47b6-9bb0-2d5a10f666ab service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:16.342831 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee63f0-f0db-47b6-9bb0-2d5a10f666ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:16.356863 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ee63f0-f0db-47b6-9bb0-2d5a10f666ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:16.358482 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ee63f0-f0db-47b6-9bb0-2d5a10f666ab service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:16.369952 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1257/2498] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:06:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:06:16.686667 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1374023-79e6-4bac-8b86-caf60a936aa9 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:16.688638 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-e1374023-79e6-4bac-8b86-caf60a936aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:16.704389 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1374023-79e6-4bac-8b86-caf60a936aa9 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:16.709997 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415e51aa-962e-493d-94b1-e0cd75f6fdf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:16.710893 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e51aa-962e-493d-94b1-e0cd75f6fdf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:16.711178 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e51aa-962e-493d-94b1-e0cd75f6fdf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:16.711457 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e51aa-962e-493d-94b1-e0cd75f6fdf6 admin admin] PATH_INFO: `/v3/projects/56709156865f4164a16dddc59eca2258` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:16.712801 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415e51aa-962e-493d-94b1-e0cd75f6fdf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56709156865f4164a16dddc59eca2258)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:16.714759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415e51aa-962e-493d-94b1-e0cd75f6fdf6 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:16.774015 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-415e51aa-962e-493d-94b1-e0cd75f6fdf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56709156865f4164a16dddc59eca2258', 'request_id': 'req-415e51aa-962e-493d-94b1-e0cd75f6fdf6'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:16.780408 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-415e51aa-962e-493d-94b1-e0cd75f6fdf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56709156865f4164a16dddc59eca2258', 'request_id': 'req-415e51aa-962e-493d-94b1-e0cd75f6fdf6'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:16.790675 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-415e51aa-962e-493d-94b1-e0cd75f6fdf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56709156865f4164a16dddc59eca2258', 'request_id': 'req-415e51aa-962e-493d-94b1-e0cd75f6fdf6'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:16.803479 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-415e51aa-962e-493d-94b1-e0cd75f6fdf6 admin admin] The token cache is being invalidate because project 56709156865f4164a16dddc59eca2258 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:06:16.803862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-415e51aa-962e-493d-94b1-e0cd75f6fdf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:16.804435 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-415e51aa-962e-493d-94b1-e0cd75f6fdf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:16.809418 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1242/2499] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:06:16 2022] DELETE /identity/v3/projects/56709156865f4164a16dddc59eca2258 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:06:16.810445 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee63f0-f0db-47b6-9bb0-2d5a10f666ab service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:16.821215 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee63f0-f0db-47b6-9bb0-2d5a10f666ab service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:16.884554 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dda7f9-10e8-420b-9f07-55c1038d85b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:16.886042 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dda7f9-10e8-420b-9f07-55c1038d85b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:16.886340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dda7f9-10e8-420b-9f07-55c1038d85b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:16.886618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dda7f9-10e8-420b-9f07-55c1038d85b2 admin admin] PATH_INFO: `/v3/users/f4bbdc66a79b457c8e306d2853d2882c` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:16.887955 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dda7f9-10e8-420b-9f07-55c1038d85b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4bbdc66a79b457c8e306d2853d2882c)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:16.889958 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dda7f9-10e8-420b-9f07-55c1038d85b2 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:16.940219 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a8dda7f9-10e8-420b-9f07-55c1038d85b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4bbdc66a79b457c8e306d2853d2882c'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:16.949182 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a8dda7f9-10e8-420b-9f07-55c1038d85b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4bbdc66a79b457c8e306d2853d2882c'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:16.956288 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a8dda7f9-10e8-420b-9f07-55c1038d85b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4bbdc66a79b457c8e306d2853d2882c'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:16.957038 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a8dda7f9-10e8-420b-9f07-55c1038d85b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4bbdc66a79b457c8e306d2853d2882c'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:16.957623 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-a8dda7f9-10e8-420b-9f07-55c1038d85b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4bbdc66a79b457c8e306d2853d2882c'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:16.977809 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1258/2500] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:06:16 2022] DELETE /identity/v3/users/f4bbdc66a79b457c8e306d2853d2882c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:06:17.271837 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415e51aa-962e-493d-94b1-e0cd75f6fdf6 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:17.273494 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-415e51aa-962e-493d-94b1-e0cd75f6fdf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:17.309483 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415e51aa-962e-493d-94b1-e0cd75f6fdf6 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:17.375731 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d082e4e-fed8-48a8-9321-80d4563b9364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:17.377081 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d082e4e-fed8-48a8-9321-80d4563b9364 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:17.377462 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d082e4e-fed8-48a8-9321-80d4563b9364 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:17.377908 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d082e4e-fed8-48a8-9321-80d4563b9364 admin admin] PATH_INFO: `/v3/projects/c29e8a6fadb0452b8abe6b0c3e119d48` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:17.379316 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d082e4e-fed8-48a8-9321-80d4563b9364 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c29e8a6fadb0452b8abe6b0c3e119d48)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:17.381328 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d082e4e-fed8-48a8-9321-80d4563b9364 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:17.435520 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-2d082e4e-fed8-48a8-9321-80d4563b9364 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c29e8a6fadb0452b8abe6b0c3e119d48', 'request_id': 'req-2d082e4e-fed8-48a8-9321-80d4563b9364'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:17.442720 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-2d082e4e-fed8-48a8-9321-80d4563b9364 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c29e8a6fadb0452b8abe6b0c3e119d48', 'request_id': 'req-2d082e4e-fed8-48a8-9321-80d4563b9364'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:17.452795 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-2d082e4e-fed8-48a8-9321-80d4563b9364 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c29e8a6fadb0452b8abe6b0c3e119d48', 'request_id': 'req-2d082e4e-fed8-48a8-9321-80d4563b9364'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:17.466854 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-2d082e4e-fed8-48a8-9321-80d4563b9364 admin admin] The token cache is being invalidate because project c29e8a6fadb0452b8abe6b0c3e119d48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:06:17.467140 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-2d082e4e-fed8-48a8-9321-80d4563b9364 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:17.467802 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-2d082e4e-fed8-48a8-9321-80d4563b9364 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:17.473113 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1243/2501] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:06:17 2022] DELETE /identity/v3/projects/c29e8a6fadb0452b8abe6b0c3e119d48 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:06:17.476157 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5954c2-985b-4c4f-98da-9d00ae4c67d8 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:17.476454 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5954c2-985b-4c4f-98da-9d00ae4c67d8 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:17.476967 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5954c2-985b-4c4f-98da-9d00ae4c67d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:17.498018 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-bd5954c2-985b-4c4f-98da-9d00ae4c67d8 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:06:17.499667 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-bd5954c2-985b-4c4f-98da-9d00ae4c67d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:17.567723 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1259/2502] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:06:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:06:17.576514 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab66d2-3b0a-4368-addf-3ea4febc35a4 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:17.576925 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab66d2-3b0a-4368-addf-3ea4febc35a4 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:17.577272 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab66d2-3b0a-4368-addf-3ea4febc35a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:17.597311 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-0bab66d2-3b0a-4368-addf-3ea4febc35a4 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:06:17.598816 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-0bab66d2-3b0a-4368-addf-3ea4febc35a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:17.607917 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1244/2503] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:06:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:06:17.612198 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5954c2-985b-4c4f-98da-9d00ae4c67d8 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:17.625002 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5954c2-985b-4c4f-98da-9d00ae4c67d8 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:17.629929 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb03dcc-004d-4307-8527-64072c1bfa4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:17.630672 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb03dcc-004d-4307-8527-64072c1bfa4b admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:17.631800 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb03dcc-004d-4307-8527-64072c1bfa4b admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:17.632178 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb03dcc-004d-4307-8527-64072c1bfa4b admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:17.633110 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb03dcc-004d-4307-8527-64072c1bfa4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 15:06:17.633490 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb03dcc-004d-4307-8527-64072c1bfa4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:17.635218 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb03dcc-004d-4307-8527-64072c1bfa4b admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:17.652432 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1260/2504] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 15:06:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 15:06:17.653705 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bab66d2-3b0a-4368-addf-3ea4febc35a4 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:17.664378 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bab66d2-3b0a-4368-addf-3ea4febc35a4 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:17.670292 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d78114-4af7-4961-b1a7-e1bc9b50fce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:17.671078 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d78114-4af7-4961-b1a7-e1bc9b50fce3 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:17.671514 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d78114-4af7-4961-b1a7-e1bc9b50fce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:17.671959 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d78114-4af7-4961-b1a7-e1bc9b50fce3 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:17.672931 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d78114-4af7-4961-b1a7-e1bc9b50fce3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:17.675150 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d78114-4af7-4961-b1a7-e1bc9b50fce3 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:17.702603 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1245/2505] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:06:17 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:06:17.706837 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb03dcc-004d-4307-8527-64072c1bfa4b admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:17.717282 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb03dcc-004d-4307-8527-64072c1bfa4b admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:17.784797 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e465bc-966c-45ce-87e8-05b84177babb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:17.785638 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e465bc-966c-45ce-87e8-05b84177babb admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:17.786059 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e465bc-966c-45ce-87e8-05b84177babb admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:17.786808 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e465bc-966c-45ce-87e8-05b84177babb admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:17.787520 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e465bc-966c-45ce-87e8-05b84177babb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:17.789879 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e465bc-966c-45ce-87e8-05b84177babb admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:17.812646 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1261/2506] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:06:17 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:06:17.816462 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d78114-4af7-4961-b1a7-e1bc9b50fce3 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:17.829340 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d78114-4af7-4961-b1a7-e1bc9b50fce3 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:17.837759 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d0494-d354-476a-bf42-d62e5b9cd9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:17.838442 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d0494-d354-476a-bf42-d62e5b9cd9ab admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:17.838719 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d0494-d354-476a-bf42-d62e5b9cd9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:17.838992 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d0494-d354-476a-bf42-d62e5b9cd9ab admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:17.839922 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d0494-d354-476a-bf42-d62e5b9cd9ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:17.841628 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d0494-d354-476a-bf42-d62e5b9cd9ab admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:17.853842 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1246/2507] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:06:17 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:06:17.857913 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e465bc-966c-45ce-87e8-05b84177babb admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:17.869177 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e465bc-966c-45ce-87e8-05b84177babb admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:17.874488 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6873d571-f311-4772-8d76-0ccb6572c4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:17.875261 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6873d571-f311-4772-8d76-0ccb6572c4fd admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:17.875656 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6873d571-f311-4772-8d76-0ccb6572c4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:17.876042 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6873d571-f311-4772-8d76-0ccb6572c4fd admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:17.877019 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6873d571-f311-4772-8d76-0ccb6572c4fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:17.879822 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6873d571-f311-4772-8d76-0ccb6572c4fd admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:17.890925 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1262/2508] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:06:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:06:17.894921 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d0494-d354-476a-bf42-d62e5b9cd9ab admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:17.906743 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d0494-d354-476a-bf42-d62e5b9cd9ab admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:17.911475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2ec02c-a739-4d4d-a306-aedb95ae409d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:17.912949 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ec02c-a739-4d4d-a306-aedb95ae409d admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:17.913334 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ec02c-a739-4d4d-a306-aedb95ae409d admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:17.913516 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ec02c-a739-4d4d-a306-aedb95ae409d admin admin] PATH_INFO: `/v3/projects/bec4e49db1804c1db9492dca3fc8a4ec/users/ab02be2568054d6187967e257196ff99/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:17.931051 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2ec02c-a739-4d4d-a306-aedb95ae409d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bec4e49db1804c1db9492dca3fc8a4ec, user_id=ab02be2568054d6187967e257196ff99, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:17.933836 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2ec02c-a739-4d4d-a306-aedb95ae409d admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:17.947455 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1247/2509] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:06:17 2022] PUT /identity/v3/projects/bec4e49db1804c1db9492dca3fc8a4ec/users/ab02be2568054d6187967e257196ff99/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 15:06:17.958801 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6a874-2eb1-408c-96cd-d412eccbace5 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:17.959493 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6a874-2eb1-408c-96cd-d412eccbace5 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:17.959493 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6a874-2eb1-408c-96cd-d412eccbace5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:17.986819 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-5ee6a874-2eb1-408c-96cd-d412eccbace5 None None] MFA Rules not processed for user `ab02be2568054d6187967e257196ff99`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:06:17.988038 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-5ee6a874-2eb1-408c-96cd-d412eccbace5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:18.057377 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1263/2510] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:06:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:06:18.058473 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2ec02c-a739-4d4d-a306-aedb95ae409d admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:18.124482 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2ec02c-a739-4d4d-a306-aedb95ae409d admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:18.129621 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ba3c53-08cf-4fe7-ac96-c4db90bd01c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:18.130329 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba3c53-08cf-4fe7-ac96-c4db90bd01c6 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:18.130603 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba3c53-08cf-4fe7-ac96-c4db90bd01c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:18.131432 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba3c53-08cf-4fe7-ac96-c4db90bd01c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:18.132150 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ba3c53-08cf-4fe7-ac96-c4db90bd01c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:18.134102 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ba3c53-08cf-4fe7-ac96-c4db90bd01c6 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:18.165389 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1248/2511] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:06:18 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 15:06:18.169872 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6a874-2eb1-408c-96cd-d412eccbace5 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:18.185087 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6a874-2eb1-408c-96cd-d412eccbace5 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:18.273257 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409cee49-bcac-4f5b-a8e9-9e7088fc5a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:18.274050 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409cee49-bcac-4f5b-a8e9-9e7088fc5a78 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:18.274404 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409cee49-bcac-4f5b-a8e9-9e7088fc5a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:18.274806 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409cee49-bcac-4f5b-a8e9-9e7088fc5a78 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:18.275800 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409cee49-bcac-4f5b-a8e9-9e7088fc5a78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:18.278382 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409cee49-bcac-4f5b-a8e9-9e7088fc5a78 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:18.306031 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1264/2512] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:06:18 2022] POST /identity/v3/users => generated 294 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:06:18.313742 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ba3c53-08cf-4fe7-ac96-c4db90bd01c6 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:18.325738 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ba3c53-08cf-4fe7-ac96-c4db90bd01c6 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:18.333397 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e76bb87-c3d1-4592-ae70-ceadb93214e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:18.334855 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76bb87-c3d1-4592-ae70-ceadb93214e3 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:18.335142 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76bb87-c3d1-4592-ae70-ceadb93214e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:18.335415 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76bb87-c3d1-4592-ae70-ceadb93214e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:18.336348 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e76bb87-c3d1-4592-ae70-ceadb93214e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:18.338990 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e76bb87-c3d1-4592-ae70-ceadb93214e3 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:18.363432 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1249/2513] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:06:18.368293 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409cee49-bcac-4f5b-a8e9-9e7088fc5a78 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:18.378179 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409cee49-bcac-4f5b-a8e9-9e7088fc5a78 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:18.383805 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d0561-25d3-4dd1-83e6-4f17de3b0cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:18.384763 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d0561-25d3-4dd1-83e6-4f17de3b0cb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:18.385049 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d0561-25d3-4dd1-83e6-4f17de3b0cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:18.385330 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d0561-25d3-4dd1-83e6-4f17de3b0cb3 admin admin] PATH_INFO: `/v3/projects/2cfa5f7c11a447809316e88eea1c2f39/users/7cc57d854afb46a5ba8e7015f7f9e95f/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:18.403350 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08d0561-25d3-4dd1-83e6-4f17de3b0cb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cfa5f7c11a447809316e88eea1c2f39, user_id=7cc57d854afb46a5ba8e7015f7f9e95f, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:18.406393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08d0561-25d3-4dd1-83e6-4f17de3b0cb3 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:18.421106 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1265/2514] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:06:18 2022] PUT /identity/v3/projects/2cfa5f7c11a447809316e88eea1c2f39/users/7cc57d854afb46a5ba8e7015f7f9e95f/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:06:18.425202 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e76bb87-c3d1-4592-ae70-ceadb93214e3 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:18.442266 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e76bb87-c3d1-4592-ae70-ceadb93214e3 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:18.505833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66911af8-ff0d-4d40-a007-68c50b9abf5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:18.506621 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66911af8-ff0d-4d40-a007-68c50b9abf5a admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:18.507069 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66911af8-ff0d-4d40-a007-68c50b9abf5a admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:18.507354 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66911af8-ff0d-4d40-a007-68c50b9abf5a admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:18.508085 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66911af8-ff0d-4d40-a007-68c50b9abf5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:18.511139 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66911af8-ff0d-4d40-a007-68c50b9abf5a admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:18.527787 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1250/2515] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:06:18.529663 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d0561-25d3-4dd1-83e6-4f17de3b0cb3 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:18.545065 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d0561-25d3-4dd1-83e6-4f17de3b0cb3 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:18.550080 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b419c81d-d1be-4e93-8f84-bc323956e476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:18.551001 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b419c81d-d1be-4e93-8f84-bc323956e476 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:18.551288 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b419c81d-d1be-4e93-8f84-bc323956e476 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:18.551565 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b419c81d-d1be-4e93-8f84-bc323956e476 admin admin] PATH_INFO: `/v3/projects/2cfa5f7c11a447809316e88eea1c2f39/users/7cc57d854afb46a5ba8e7015f7f9e95f/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:18.555824 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b419c81d-d1be-4e93-8f84-bc323956e476 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cfa5f7c11a447809316e88eea1c2f39, user_id=7cc57d854afb46a5ba8e7015f7f9e95f, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:18.558821 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b419c81d-d1be-4e93-8f84-bc323956e476 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:18.573720 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1266/2516] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:06:18 2022] PUT /identity/v3/projects/2cfa5f7c11a447809316e88eea1c2f39/users/7cc57d854afb46a5ba8e7015f7f9e95f/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:06:18.583681 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a36fc6-9230-49b3-a32f-84a47a850587 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:18.584231 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a36fc6-9230-49b3-a32f-84a47a850587 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:18.584484 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a36fc6-9230-49b3-a32f-84a47a850587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:18.612061 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-f9a36fc6-9230-49b3-a32f-84a47a850587 None None] MFA Rules not processed for user `7cc57d854afb46a5ba8e7015f7f9e95f`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:06:18.613247 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-f9a36fc6-9230-49b3-a32f-84a47a850587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:18.693367 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1251/2517] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:06:18 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:06:18.704532 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b419c81d-d1be-4e93-8f84-bc323956e476 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:18.707340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-b419c81d-d1be-4e93-8f84-bc323956e476 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:18.783841 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b419c81d-d1be-4e93-8f84-bc323956e476 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:18.790823 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd23a4b-8f13-4695-baa8-128aa0562f33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:18.791583 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd23a4b-8f13-4695-baa8-128aa0562f33 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:18.791933 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd23a4b-8f13-4695-baa8-128aa0562f33 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:18.792325 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd23a4b-8f13-4695-baa8-128aa0562f33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:18.869601 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd23a4b-8f13-4695-baa8-128aa0562f33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:18.871714 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd23a4b-8f13-4695-baa8-128aa0562f33 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:18.895613 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1267/2518] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:06:18 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:06:21.493904 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a36fc6-9230-49b3-a32f-84a47a850587 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:21.561365 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a36fc6-9230-49b3-a32f-84a47a850587 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:21.566076 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a97038-32e9-4861-8616-cbbf94b8e04d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:21.566975 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a97038-32e9-4861-8616-cbbf94b8e04d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:21.567254 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a97038-32e9-4861-8616-cbbf94b8e04d admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:21.567525 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a97038-32e9-4861-8616-cbbf94b8e04d admin admin] PATH_INFO: `/v3/users/ab02be2568054d6187967e257196ff99` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:21.568808 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a97038-32e9-4861-8616-cbbf94b8e04d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab02be2568054d6187967e257196ff99)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:21.571335 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a97038-32e9-4861-8616-cbbf94b8e04d admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:21.621524 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-68a97038-32e9-4861-8616-cbbf94b8e04d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab02be2568054d6187967e257196ff99'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:21.622113 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-68a97038-32e9-4861-8616-cbbf94b8e04d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab02be2568054d6187967e257196ff99'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:21.638552 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-68a97038-32e9-4861-8616-cbbf94b8e04d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab02be2568054d6187967e257196ff99'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:21.646551 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-68a97038-32e9-4861-8616-cbbf94b8e04d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab02be2568054d6187967e257196ff99'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:21.647319 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-68a97038-32e9-4861-8616-cbbf94b8e04d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab02be2568054d6187967e257196ff99'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:21.666475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1252/2519] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:06:21 2022] DELETE /identity/v3/users/ab02be2568054d6187967e257196ff99 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:06:21.671230 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11451b07-9039-4754-b3a7-4e2522b280e2 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:21.671844 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11451b07-9039-4754-b3a7-4e2522b280e2 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:21.672054 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11451b07-9039-4754-b3a7-4e2522b280e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:21.690943 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-11451b07-9039-4754-b3a7-4e2522b280e2 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:06:21.692165 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-11451b07-9039-4754-b3a7-4e2522b280e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:21.754036 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1268/2520] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:06:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:06:21.764966 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a97038-32e9-4861-8616-cbbf94b8e04d admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:21.766405 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-68a97038-32e9-4861-8616-cbbf94b8e04d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:21.842330 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a97038-32e9-4861-8616-cbbf94b8e04d admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:21.847434 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02362e18-9011-423e-b4b0-6f75866ba6c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:21.848242 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02362e18-9011-423e-b4b0-6f75866ba6c6 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:21.848571 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02362e18-9011-423e-b4b0-6f75866ba6c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:21.848972 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02362e18-9011-423e-b4b0-6f75866ba6c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:21.860188 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02362e18-9011-423e-b4b0-6f75866ba6c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:21.861918 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02362e18-9011-423e-b4b0-6f75866ba6c6 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:21.874089 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1253/2521] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:06:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:06:22.347088 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11451b07-9039-4754-b3a7-4e2522b280e2 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:22.348527 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-11451b07-9039-4754-b3a7-4e2522b280e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:22.365861 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11451b07-9039-4754-b3a7-4e2522b280e2 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:22.371814 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad89cbd-c1a4-4069-88cc-d97d328ddf27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:22.372750 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad89cbd-c1a4-4069-88cc-d97d328ddf27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:22.373036 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad89cbd-c1a4-4069-88cc-d97d328ddf27 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:22.373318 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad89cbd-c1a4-4069-88cc-d97d328ddf27 admin admin] PATH_INFO: `/v3/projects/bec4e49db1804c1db9492dca3fc8a4ec` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:22.374569 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad89cbd-c1a4-4069-88cc-d97d328ddf27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bec4e49db1804c1db9492dca3fc8a4ec)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:22.376576 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad89cbd-c1a4-4069-88cc-d97d328ddf27 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:22.442192 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-5ad89cbd-c1a4-4069-88cc-d97d328ddf27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bec4e49db1804c1db9492dca3fc8a4ec', 'request_id': 'req-5ad89cbd-c1a4-4069-88cc-d97d328ddf27'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:22.448429 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-5ad89cbd-c1a4-4069-88cc-d97d328ddf27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bec4e49db1804c1db9492dca3fc8a4ec', 'request_id': 'req-5ad89cbd-c1a4-4069-88cc-d97d328ddf27'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:22.457134 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-5ad89cbd-c1a4-4069-88cc-d97d328ddf27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bec4e49db1804c1db9492dca3fc8a4ec', 'request_id': 'req-5ad89cbd-c1a4-4069-88cc-d97d328ddf27'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:22.471106 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-5ad89cbd-c1a4-4069-88cc-d97d328ddf27 admin admin] The token cache is being invalidate because project bec4e49db1804c1db9492dca3fc8a4ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:06:22.471494 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-5ad89cbd-c1a4-4069-88cc-d97d328ddf27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:22.472222 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-5ad89cbd-c1a4-4069-88cc-d97d328ddf27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:22.474160 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1269/2522] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:06:22 2022] DELETE /identity/v3/projects/bec4e49db1804c1db9492dca3fc8a4ec => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:06:22.478345 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02362e18-9011-423e-b4b0-6f75866ba6c6 service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:22.490913 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02362e18-9011-423e-b4b0-6f75866ba6c6 service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:22.556789 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a235a128-e5a5-4797-91dc-9012393d783c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:22.558503 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a235a128-e5a5-4797-91dc-9012393d783c admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:22.559213 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a235a128-e5a5-4797-91dc-9012393d783c admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:22.559714 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a235a128-e5a5-4797-91dc-9012393d783c admin admin] PATH_INFO: `/v3/users/7cc57d854afb46a5ba8e7015f7f9e95f` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:22.561232 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a235a128-e5a5-4797-91dc-9012393d783c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cc57d854afb46a5ba8e7015f7f9e95f)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:22.563365 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a235a128-e5a5-4797-91dc-9012393d783c admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:22.565734 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad89cbd-c1a4-4069-88cc-d97d328ddf27 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:22.567659 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-5ad89cbd-c1a4-4069-88cc-d97d328ddf27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:22.597446 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad89cbd-c1a4-4069-88cc-d97d328ddf27 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:22.604367 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4383fd2d-6d3f-4307-a64c-2ce659db0c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:22.605349 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4383fd2d-6d3f-4307-a64c-2ce659db0c30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:22.605638 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4383fd2d-6d3f-4307-a64c-2ce659db0c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:22.605898 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4383fd2d-6d3f-4307-a64c-2ce659db0c30 admin admin] PATH_INFO: `/v3/users/22199a431b8a42789bbc0f5e30b816ec` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:22.607158 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4383fd2d-6d3f-4307-a64c-2ce659db0c30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22199a431b8a42789bbc0f5e30b816ec)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:22.609110 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4383fd2d-6d3f-4307-a64c-2ce659db0c30 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:22.623460 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-a235a128-e5a5-4797-91dc-9012393d783c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cc57d854afb46a5ba8e7015f7f9e95f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:22.624119 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-a235a128-e5a5-4797-91dc-9012393d783c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cc57d854afb46a5ba8e7015f7f9e95f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:22.631220 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-a235a128-e5a5-4797-91dc-9012393d783c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cc57d854afb46a5ba8e7015f7f9e95f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:22.640466 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-a235a128-e5a5-4797-91dc-9012393d783c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cc57d854afb46a5ba8e7015f7f9e95f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:22.641549 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-a235a128-e5a5-4797-91dc-9012393d783c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cc57d854afb46a5ba8e7015f7f9e95f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:22.659311 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-4383fd2d-6d3f-4307-a64c-2ce659db0c30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22199a431b8a42789bbc0f5e30b816ec'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:22.661981 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1254/2523] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:06:22 2022] DELETE /identity/v3/users/7cc57d854afb46a5ba8e7015f7f9e95f => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:06:22.670452 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-4383fd2d-6d3f-4307-a64c-2ce659db0c30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22199a431b8a42789bbc0f5e30b816ec'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:22.678859 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-4383fd2d-6d3f-4307-a64c-2ce659db0c30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22199a431b8a42789bbc0f5e30b816ec'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:22.680030 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-4383fd2d-6d3f-4307-a64c-2ce659db0c30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22199a431b8a42789bbc0f5e30b816ec'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:22.680702 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-4383fd2d-6d3f-4307-a64c-2ce659db0c30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22199a431b8a42789bbc0f5e30b816ec'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:22.698499 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1270/2524] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:06:22 2022] DELETE /identity/v3/users/22199a431b8a42789bbc0f5e30b816ec => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 15:06:22.878988 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a235a128-e5a5-4797-91dc-9012393d783c admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:22.880803 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-a235a128-e5a5-4797-91dc-9012393d783c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:22.950835 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a235a128-e5a5-4797-91dc-9012393d783c admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:22.960400 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4383fd2d-6d3f-4307-a64c-2ce659db0c30 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:22.962518 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-4383fd2d-6d3f-4307-a64c-2ce659db0c30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:23.011148 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff423ce-4300-4054-b67c-175f355c25a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:23.012025 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff423ce-4300-4054-b67c-175f355c25a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:23.012303 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff423ce-4300-4054-b67c-175f355c25a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:23.012576 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff423ce-4300-4054-b67c-175f355c25a6 admin admin] PATH_INFO: `/v3/projects/4be2354633354d249a0598f589a987b8` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:23.013821 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff423ce-4300-4054-b67c-175f355c25a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4be2354633354d249a0598f589a987b8)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:23.015751 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff423ce-4300-4054-b67c-175f355c25a6 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:23.048393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4383fd2d-6d3f-4307-a64c-2ce659db0c30 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:23.062225 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-7ff423ce-4300-4054-b67c-175f355c25a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4be2354633354d249a0598f589a987b8', 'request_id': 'req-7ff423ce-4300-4054-b67c-175f355c25a6'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:23.067892 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-7ff423ce-4300-4054-b67c-175f355c25a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4be2354633354d249a0598f589a987b8', 'request_id': 'req-7ff423ce-4300-4054-b67c-175f355c25a6'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:23.076750 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-7ff423ce-4300-4054-b67c-175f355c25a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4be2354633354d249a0598f589a987b8', 'request_id': 'req-7ff423ce-4300-4054-b67c-175f355c25a6'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:23.088808 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-7ff423ce-4300-4054-b67c-175f355c25a6 admin admin] The token cache is being invalidate because project 4be2354633354d249a0598f589a987b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:06:23.089202 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-7ff423ce-4300-4054-b67c-175f355c25a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:23.089886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-7ff423ce-4300-4054-b67c-175f355c25a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:23.091438 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1255/2525] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:06:22 2022] DELETE /identity/v3/projects/4be2354633354d249a0598f589a987b8 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:06:23.095441 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff423ce-4300-4054-b67c-175f355c25a6 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:23.096809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-7ff423ce-4300-4054-b67c-175f355c25a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:23.111054 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c816dab9-28cc-4e8b-9067-df9543510953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:23.111928 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816dab9-28cc-4e8b-9067-df9543510953 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:23.112212 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816dab9-28cc-4e8b-9067-df9543510953 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:23.112492 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816dab9-28cc-4e8b-9067-df9543510953 admin admin] PATH_INFO: `/v3/projects/2cfa5f7c11a447809316e88eea1c2f39` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:23.113721 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c816dab9-28cc-4e8b-9067-df9543510953 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cfa5f7c11a447809316e88eea1c2f39)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:23.115622 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c816dab9-28cc-4e8b-9067-df9543510953 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:23.162893 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff423ce-4300-4054-b67c-175f355c25a6 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:23.165904 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c816dab9-28cc-4e8b-9067-df9543510953 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cfa5f7c11a447809316e88eea1c2f39', 'request_id': 'req-c816dab9-28cc-4e8b-9067-df9543510953'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:23.171563 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c816dab9-28cc-4e8b-9067-df9543510953 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cfa5f7c11a447809316e88eea1c2f39', 'request_id': 'req-c816dab9-28cc-4e8b-9067-df9543510953'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:23.180172 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c816dab9-28cc-4e8b-9067-df9543510953 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cfa5f7c11a447809316e88eea1c2f39', 'request_id': 'req-c816dab9-28cc-4e8b-9067-df9543510953'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:23.191409 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c816dab9-28cc-4e8b-9067-df9543510953 admin admin] The token cache is being invalidate because project 2cfa5f7c11a447809316e88eea1c2f39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:06:23.191699 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c816dab9-28cc-4e8b-9067-df9543510953 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:23.192351 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c816dab9-28cc-4e8b-9067-df9543510953 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:23.193946 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1271/2526] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:06:22 2022] DELETE /identity/v3/projects/2cfa5f7c11a447809316e88eea1c2f39 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 15:06:23.198777 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4515b3-375a-4a8e-9d6a-cee746ac5a34 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:23.199070 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4515b3-375a-4a8e-9d6a-cee746ac5a34 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:23.199348 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4515b3-375a-4a8e-9d6a-cee746ac5a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:23.217624 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-ae4515b3-375a-4a8e-9d6a-cee746ac5a34 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:06:23.218842 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-ae4515b3-375a-4a8e-9d6a-cee746ac5a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:23.225015 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eac6c42-0d96-4737-9e6b-00321dabb28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:23.225895 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac6c42-0d96-4737-9e6b-00321dabb28f admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:23.227324 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac6c42-0d96-4737-9e6b-00321dabb28f admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:23.227633 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac6c42-0d96-4737-9e6b-00321dabb28f admin admin] PATH_INFO: `/v3/users/667e72222b0b4a9eadfd358358070782` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:23.228941 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eac6c42-0d96-4737-9e6b-00321dabb28f admin admin] RBAC: Authorizing `identity:delete_user(user_id=667e72222b0b4a9eadfd358358070782)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:23.231036 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eac6c42-0d96-4737-9e6b-00321dabb28f admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:23.284025 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1272/2527] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:06:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 15:06:23.285544 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8eac6c42-0d96-4737-9e6b-00321dabb28f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '667e72222b0b4a9eadfd358358070782'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:23.286211 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8eac6c42-0d96-4737-9e6b-00321dabb28f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '667e72222b0b4a9eadfd358358070782'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:23.294100 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b4caac-64d4-44da-a871-2c222b980cbb None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:23.294409 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b4caac-64d4-44da-a871-2c222b980cbb None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:23.294553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8eac6c42-0d96-4737-9e6b-00321dabb28f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '667e72222b0b4a9eadfd358358070782'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:23.294845 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b4caac-64d4-44da-a871-2c222b980cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:23.302555 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8eac6c42-0d96-4737-9e6b-00321dabb28f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '667e72222b0b4a9eadfd358358070782'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:23.302555 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8eac6c42-0d96-4737-9e6b-00321dabb28f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '667e72222b0b4a9eadfd358358070782'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:23.314600 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-41b4caac-64d4-44da-a871-2c222b980cbb None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:06:23.315816 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-41b4caac-64d4-44da-a871-2c222b980cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:23.324565 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1273/2528] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:06:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 15:06:23.327840 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1256/2529] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:06:23 2022] DELETE /identity/v3/users/667e72222b0b4a9eadfd358358070782 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 15:06:23.329053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b4caac-64d4-44da-a871-2c222b980cbb None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:23.339412 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b4caac-64d4-44da-a871-2c222b980cbb None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:23.399296 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8630463d-ad2e-4c23-823f-e57845437a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:23.400109 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630463d-ad2e-4c23-823f-e57845437a76 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:23.400397 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630463d-ad2e-4c23-823f-e57845437a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:23.400710 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630463d-ad2e-4c23-823f-e57845437a76 admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:23.401452 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8630463d-ad2e-4c23-823f-e57845437a76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 15:06:23.402045 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8630463d-ad2e-4c23-823f-e57845437a76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:23.403705 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8630463d-ad2e-4c23-823f-e57845437a76 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:23.419452 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1274/2530] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 15:06:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 15:06:23.420516 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eac6c42-0d96-4737-9e6b-00321dabb28f admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:23.430629 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eac6c42-0d96-4737-9e6b-00321dabb28f admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:23.435312 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960cc282-52c8-4dfa-9148-1fe6cebb678e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:23.435972 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960cc282-52c8-4dfa-9148-1fe6cebb678e admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:23.436256 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960cc282-52c8-4dfa-9148-1fe6cebb678e admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:23.436527 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960cc282-52c8-4dfa-9148-1fe6cebb678e admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:23.437246 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960cc282-52c8-4dfa-9148-1fe6cebb678e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:23.439148 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960cc282-52c8-4dfa-9148-1fe6cebb678e admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:23.466857 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1257/2531] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:06:23 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:06:23.471248 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8630463d-ad2e-4c23-823f-e57845437a76 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:23.482385 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8630463d-ad2e-4c23-823f-e57845437a76 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:23.505844 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960cc282-52c8-4dfa-9148-1fe6cebb678e admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:23.508063 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-960cc282-52c8-4dfa-9148-1fe6cebb678e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:23.559005 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1740b78-542e-45b7-8481-1033179fe238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:23.559801 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1740b78-542e-45b7-8481-1033179fe238 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:23.560155 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1740b78-542e-45b7-8481-1033179fe238 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:23.560439 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1740b78-542e-45b7-8481-1033179fe238 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:23.561220 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1740b78-542e-45b7-8481-1033179fe238 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:23.563183 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1740b78-542e-45b7-8481-1033179fe238 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:23.583873 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1275/2532] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:06:23 2022] POST /identity/v3/users => generated 287 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:06:23.585963 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960cc282-52c8-4dfa-9148-1fe6cebb678e admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:23.588344 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1740b78-542e-45b7-8481-1033179fe238 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:23.591025 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d4ce84-b40e-43f1-b790-ab64835c4a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:23.591896 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4ce84-b40e-43f1-b790-ab64835c4a45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:23.592178 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4ce84-b40e-43f1-b790-ab64835c4a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:23.592449 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4ce84-b40e-43f1-b790-ab64835c4a45 admin admin] PATH_INFO: `/v3/projects/2e9600e4323648cd9c7794d67bce9fe3` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:23.593683 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d4ce84-b40e-43f1-b790-ab64835c4a45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e9600e4323648cd9c7794d67bce9fe3)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:23.595593 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d4ce84-b40e-43f1-b790-ab64835c4a45 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:23.598030 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1740b78-542e-45b7-8481-1033179fe238 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:23.603286 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfac6ca5-7f77-4283-aca6-815ad3fd90ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:23.603944 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac6ca5-7f77-4283-aca6-815ad3fd90ca admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:23.604225 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac6ca5-7f77-4283-aca6-815ad3fd90ca admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:23.604494 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac6ca5-7f77-4283-aca6-815ad3fd90ca admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:23.605248 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfac6ca5-7f77-4283-aca6-815ad3fd90ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:23.606821 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfac6ca5-7f77-4283-aca6-815ad3fd90ca admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:23.620687 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1276/2533] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 06 15:06:23.622056 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfac6ca5-7f77-4283-aca6-815ad3fd90ca admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:23.633148 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfac6ca5-7f77-4283-aca6-815ad3fd90ca admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:23.644112 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-04d4ce84-b40e-43f1-b790-ab64835c4a45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e9600e4323648cd9c7794d67bce9fe3', 'request_id': 'req-04d4ce84-b40e-43f1-b790-ab64835c4a45'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:23.649702 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-04d4ce84-b40e-43f1-b790-ab64835c4a45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e9600e4323648cd9c7794d67bce9fe3', 'request_id': 'req-04d4ce84-b40e-43f1-b790-ab64835c4a45'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:23.657018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-04d4ce84-b40e-43f1-b790-ab64835c4a45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e9600e4323648cd9c7794d67bce9fe3', 'request_id': 'req-04d4ce84-b40e-43f1-b790-ab64835c4a45'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:23.669947 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-04d4ce84-b40e-43f1-b790-ab64835c4a45 admin admin] The token cache is being invalidate because project 2e9600e4323648cd9c7794d67bce9fe3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:06:23.670696 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-04d4ce84-b40e-43f1-b790-ab64835c4a45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:23.671584 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-04d4ce84-b40e-43f1-b790-ab64835c4a45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:23.676683 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1258/2534] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:06:23 2022] DELETE /identity/v3/projects/2e9600e4323648cd9c7794d67bce9fe3 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:06:23.701081 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f282f-9cdb-40b4-a7c4-9a3830cee745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:23.702580 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f282f-9cdb-40b4-a7c4-9a3830cee745 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:23.703126 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f282f-9cdb-40b4-a7c4-9a3830cee745 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:23.703632 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f282f-9cdb-40b4-a7c4-9a3830cee745 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:23.704666 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646f282f-9cdb-40b4-a7c4-9a3830cee745 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:23.706573 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646f282f-9cdb-40b4-a7c4-9a3830cee745 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:23.719048 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1277/2535] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 06 15:06:23.723463 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d4ce84-b40e-43f1-b790-ab64835c4a45 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:23.725343 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-04d4ce84-b40e-43f1-b790-ab64835c4a45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:23.740320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d4ce84-b40e-43f1-b790-ab64835c4a45 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:23.748153 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3f7dd4-4952-4726-b052-77ccaffa1b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:23.749682 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f7dd4-4952-4726-b052-77ccaffa1b79 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:23.750079 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f7dd4-4952-4726-b052-77ccaffa1b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:23.750514 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f7dd4-4952-4726-b052-77ccaffa1b79 admin admin] PATH_INFO: `/v3/projects/11458502b0ed41fa9d2d6e41ec6efdb1/users/b73f55d6e074451cb86c5df36b98fa2f/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:23.768659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3f7dd4-4952-4726-b052-77ccaffa1b79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11458502b0ed41fa9d2d6e41ec6efdb1, user_id=b73f55d6e074451cb86c5df36b98fa2f, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:23.771646 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3f7dd4-4952-4726-b052-77ccaffa1b79 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:23.788868 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1259/2536] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:06:23 2022] PUT /identity/v3/projects/11458502b0ed41fa9d2d6e41ec6efdb1/users/b73f55d6e074451cb86c5df36b98fa2f/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 15:06:23.800758 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b23d9-daab-4eda-8de8-36e30979c273 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:23.801206 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b23d9-daab-4eda-8de8-36e30979c273 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:23.801588 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b23d9-daab-4eda-8de8-36e30979c273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:23.833199 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-db9b23d9-daab-4eda-8de8-36e30979c273 None None] MFA Rules not processed for user `b73f55d6e074451cb86c5df36b98fa2f`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:06:23.834570 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-db9b23d9-daab-4eda-8de8-36e30979c273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:23.908578 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1278/2537] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:06:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:06:23.921066 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3f7dd4-4952-4726-b052-77ccaffa1b79 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:23.922717 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-5b3f7dd4-4952-4726-b052-77ccaffa1b79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:23.981562 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3f7dd4-4952-4726-b052-77ccaffa1b79 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:23.985172 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a7007-01ee-4a20-8ed6-75d0c91bddc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:23.985819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a7007-01ee-4a20-8ed6-75d0c91bddc2 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:23.986100 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a7007-01ee-4a20-8ed6-75d0c91bddc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:23.986376 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a7007-01ee-4a20-8ed6-75d0c91bddc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:23.995383 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967a7007-01ee-4a20-8ed6-75d0c91bddc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:23.996850 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967a7007-01ee-4a20-8ed6-75d0c91bddc2 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:24.005552 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1260/2538] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:06:23 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:06:24.033193 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b23d9-daab-4eda-8de8-36e30979c273 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:24.097216 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b23d9-daab-4eda-8de8-36e30979c273 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:24.102649 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c2131e-7caa-4410-9e4e-1f78939b3ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:24.103559 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2131e-7caa-4410-9e4e-1f78939b3ad6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:24.103840 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2131e-7caa-4410-9e4e-1f78939b3ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:24.104114 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2131e-7caa-4410-9e4e-1f78939b3ad6 admin admin] PATH_INFO: `/v3/users/b73f55d6e074451cb86c5df36b98fa2f` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:24.105334 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c2131e-7caa-4410-9e4e-1f78939b3ad6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b73f55d6e074451cb86c5df36b98fa2f)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:24.107334 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c2131e-7caa-4410-9e4e-1f78939b3ad6 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:24.160421 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-88c2131e-7caa-4410-9e4e-1f78939b3ad6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b73f55d6e074451cb86c5df36b98fa2f'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:24.169101 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-88c2131e-7caa-4410-9e4e-1f78939b3ad6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b73f55d6e074451cb86c5df36b98fa2f'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:24.176161 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-88c2131e-7caa-4410-9e4e-1f78939b3ad6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b73f55d6e074451cb86c5df36b98fa2f'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:24.176768 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-88c2131e-7caa-4410-9e4e-1f78939b3ad6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b73f55d6e074451cb86c5df36b98fa2f'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:24.177384 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-88c2131e-7caa-4410-9e4e-1f78939b3ad6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b73f55d6e074451cb86c5df36b98fa2f'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:24.194016 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1279/2539] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:06:24 2022] DELETE /identity/v3/users/b73f55d6e074451cb86c5df36b98fa2f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:06:24.198831 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d261e4-f53c-41e4-abb6-0169f8296437 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:24.199169 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d261e4-f53c-41e4-abb6-0169f8296437 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:24.199458 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d261e4-f53c-41e4-abb6-0169f8296437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:24.217693 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-52d261e4-f53c-41e4-abb6-0169f8296437 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:06:24.218889 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-52d261e4-f53c-41e4-abb6-0169f8296437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:24.277484 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1261/2540] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:06:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:06:24.287363 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c2131e-7caa-4410-9e4e-1f78939b3ad6 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:24.289413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-88c2131e-7caa-4410-9e4e-1f78939b3ad6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:24.345441 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c2131e-7caa-4410-9e4e-1f78939b3ad6 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:24.349006 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13ac95-4b6a-45fa-8fa1-4c708de36db4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:24.349649 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13ac95-4b6a-45fa-8fa1-4c708de36db4 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:24.349928 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13ac95-4b6a-45fa-8fa1-4c708de36db4 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:24.350199 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13ac95-4b6a-45fa-8fa1-4c708de36db4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:24.359935 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d13ac95-4b6a-45fa-8fa1-4c708de36db4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:24.361417 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d13ac95-4b6a-45fa-8fa1-4c708de36db4 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:24.371582 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1280/2541] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:06:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:06:24.458968 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d261e4-f53c-41e4-abb6-0169f8296437 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:24.461681 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-52d261e4-f53c-41e4-abb6-0169f8296437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:24.545426 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d261e4-f53c-41e4-abb6-0169f8296437 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:24.551982 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aa1839-246b-41aa-a20e-f97042b59463 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887a619ef5b5417db98ff15a9f712724', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:24.552809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa1839-246b-41aa-a20e-f97042b59463 service placement] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:24.553224 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa1839-246b-41aa-a20e-f97042b59463 service placement] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:24.553601 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa1839-246b-41aa-a20e-f97042b59463 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:24.556223 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d8aa1839-246b-41aa-a20e-f97042b59463 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:24.574098 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aa1839-246b-41aa-a20e-f97042b59463 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:24.574098 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aa1839-246b-41aa-a20e-f97042b59463 service placement] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:24.583727 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1262/2542] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:06:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:06:24.667088 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13ac95-4b6a-45fa-8fa1-4c708de36db4 service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:24.669277 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-0d13ac95-4b6a-45fa-8fa1-4c708de36db4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:24.687672 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13ac95-4b6a-45fa-8fa1-4c708de36db4 service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:24.692718 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f8b9a-7333-4c9d-b16a-78761eaf3f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:24.693520 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f8b9a-7333-4c9d-b16a-78761eaf3f80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:24.696023 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f8b9a-7333-4c9d-b16a-78761eaf3f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:24.696023 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f8b9a-7333-4c9d-b16a-78761eaf3f80 admin admin] PATH_INFO: `/v3/projects/11458502b0ed41fa9d2d6e41ec6efdb1` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:24.699052 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80f8b9a-7333-4c9d-b16a-78761eaf3f80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11458502b0ed41fa9d2d6e41ec6efdb1)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:24.701071 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80f8b9a-7333-4c9d-b16a-78761eaf3f80 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:24.751279 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c80f8b9a-7333-4c9d-b16a-78761eaf3f80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11458502b0ed41fa9d2d6e41ec6efdb1', 'request_id': 'req-c80f8b9a-7333-4c9d-b16a-78761eaf3f80'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:24.756585 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c80f8b9a-7333-4c9d-b16a-78761eaf3f80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11458502b0ed41fa9d2d6e41ec6efdb1', 'request_id': 'req-c80f8b9a-7333-4c9d-b16a-78761eaf3f80'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:24.767390 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c80f8b9a-7333-4c9d-b16a-78761eaf3f80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11458502b0ed41fa9d2d6e41ec6efdb1', 'request_id': 'req-c80f8b9a-7333-4c9d-b16a-78761eaf3f80'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:24.782496 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c80f8b9a-7333-4c9d-b16a-78761eaf3f80 admin admin] The token cache is being invalidate because project 11458502b0ed41fa9d2d6e41ec6efdb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:06:24.782881 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c80f8b9a-7333-4c9d-b16a-78761eaf3f80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:24.783584 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c80f8b9a-7333-4c9d-b16a-78761eaf3f80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:24.785188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1281/2543] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:06:24 2022] DELETE /identity/v3/projects/11458502b0ed41fa9d2d6e41ec6efdb1 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:06:24.790845 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5795d0-3eae-4a52-a4ab-7b81ae38c180 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:24.791486 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5795d0-3eae-4a52-a4ab-7b81ae38c180 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:24.791486 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5795d0-3eae-4a52-a4ab-7b81ae38c180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:24.812249 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-4a5795d0-3eae-4a52-a4ab-7b81ae38c180 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:06:24.813492 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-4a5795d0-3eae-4a52-a4ab-7b81ae38c180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:24.877147 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1263/2544] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:06:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:06:24.885374 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6799d-287f-4cee-94c6-bc9750ccf2a4 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:24.885668 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6799d-287f-4cee-94c6-bc9750ccf2a4 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:24.885951 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6799d-287f-4cee-94c6-bc9750ccf2a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:24.907245 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-95d6799d-287f-4cee-94c6-bc9750ccf2a4 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:06:24.908485 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-95d6799d-287f-4cee-94c6-bc9750ccf2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:24.917184 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1282/2545] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:06:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:06:24.921472 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5795d0-3eae-4a52-a4ab-7b81ae38c180 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:24.931736 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5795d0-3eae-4a52-a4ab-7b81ae38c180 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:24.937679 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e4adae-422a-4b81-bbb9-8a0f472f265b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:24.938332 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4adae-422a-4b81-bbb9-8a0f472f265b admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:24.938608 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4adae-422a-4b81-bbb9-8a0f472f265b admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:24.938883 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4adae-422a-4b81-bbb9-8a0f472f265b admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:24.939596 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e4adae-422a-4b81-bbb9-8a0f472f265b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 15:06:24.939968 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e4adae-422a-4b81-bbb9-8a0f472f265b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:24.941579 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e4adae-422a-4b81-bbb9-8a0f472f265b admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:24.954537 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1264/2546] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 15:06:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 15:06:24.958939 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6799d-287f-4cee-94c6-bc9750ccf2a4 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:24.969120 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6799d-287f-4cee-94c6-bc9750ccf2a4 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:24.973841 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856f9507-559a-4992-91e8-578b65536e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:24.974501 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856f9507-559a-4992-91e8-578b65536e45 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:24.974789 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856f9507-559a-4992-91e8-578b65536e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:24.975069 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856f9507-559a-4992-91e8-578b65536e45 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:24.975755 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856f9507-559a-4992-91e8-578b65536e45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:24.978256 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856f9507-559a-4992-91e8-578b65536e45 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:25.008001 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1283/2547] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:06:24 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:06:25.011844 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e4adae-422a-4b81-bbb9-8a0f472f265b admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:25.021758 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e4adae-422a-4b81-bbb9-8a0f472f265b admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:25.085841 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e9315-67e2-480d-a0e9-992193bcbd6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:25.086535 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e9315-67e2-480d-a0e9-992193bcbd6a admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:25.086819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e9315-67e2-480d-a0e9-992193bcbd6a admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:25.087097 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e9315-67e2-480d-a0e9-992193bcbd6a admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:25.087788 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2e9315-67e2-480d-a0e9-992193bcbd6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:25.089819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2e9315-67e2-480d-a0e9-992193bcbd6a admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:25.110886 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1265/2548] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:06:25 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:06:25.115064 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856f9507-559a-4992-91e8-578b65536e45 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:25.127731 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856f9507-559a-4992-91e8-578b65536e45 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:25.134509 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196546c2-3bb5-4bf9-b2fd-af62e12ea122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:25.136079 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196546c2-3bb5-4bf9-b2fd-af62e12ea122 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:25.136079 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196546c2-3bb5-4bf9-b2fd-af62e12ea122 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:25.136079 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196546c2-3bb5-4bf9-b2fd-af62e12ea122 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:25.136460 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196546c2-3bb5-4bf9-b2fd-af62e12ea122 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:25.138151 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196546c2-3bb5-4bf9-b2fd-af62e12ea122 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:25.156739 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1284/2549] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:06:25.160584 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e9315-67e2-480d-a0e9-992193bcbd6a admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:25.172021 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e9315-67e2-480d-a0e9-992193bcbd6a admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:25.178318 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fadfdf-9ae1-4647-a411-58c0424c8392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:25.179042 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fadfdf-9ae1-4647-a411-58c0424c8392 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:25.180441 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fadfdf-9ae1-4647-a411-58c0424c8392 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:25.180755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fadfdf-9ae1-4647-a411-58c0424c8392 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:25.181489 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fadfdf-9ae1-4647-a411-58c0424c8392 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:25.183088 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fadfdf-9ae1-4647-a411-58c0424c8392 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:25.193350 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1266/2550] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 06 15:06:25.197904 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196546c2-3bb5-4bf9-b2fd-af62e12ea122 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:25.209176 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196546c2-3bb5-4bf9-b2fd-af62e12ea122 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:25.214380 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48c3b02-3c23-431a-a1a3-89b08b8e7f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:25.215280 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c3b02-3c23-431a-a1a3-89b08b8e7f8f admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:25.215618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c3b02-3c23-431a-a1a3-89b08b8e7f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:25.215874 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c3b02-3c23-431a-a1a3-89b08b8e7f8f admin admin] PATH_INFO: `/v3/projects/ea4d04cd80ad465a8aa19d5ac5bcab55/users/d7047dd629cc4e23a7fc3b2e49b2d60f/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:25.232841 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48c3b02-3c23-431a-a1a3-89b08b8e7f8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea4d04cd80ad465a8aa19d5ac5bcab55, user_id=d7047dd629cc4e23a7fc3b2e49b2d60f, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:25.235522 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48c3b02-3c23-431a-a1a3-89b08b8e7f8f admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:25.249891 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1285/2551] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:06:25 2022] PUT /identity/v3/projects/ea4d04cd80ad465a8aa19d5ac5bcab55/users/d7047dd629cc4e23a7fc3b2e49b2d60f/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:06:25.261846 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e6eb7-68c0-4285-95b1-76f6430e3f4c None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:25.262060 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e6eb7-68c0-4285-95b1-76f6430e3f4c None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:25.262454 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e6eb7-68c0-4285-95b1-76f6430e3f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:25.291667 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-843e6eb7-68c0-4285-95b1-76f6430e3f4c None None] MFA Rules not processed for user `d7047dd629cc4e23a7fc3b2e49b2d60f`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:06:25.292921 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-843e6eb7-68c0-4285-95b1-76f6430e3f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:25.357837 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1267/2552] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:06:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:06:25.368743 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48c3b02-3c23-431a-a1a3-89b08b8e7f8f admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:25.370499 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e48c3b02-3c23-431a-a1a3-89b08b8e7f8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:25.429660 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48c3b02-3c23-431a-a1a3-89b08b8e7f8f admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:25.433423 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f988cf07-599b-48d9-a6b8-69ec797279b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:25.434079 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988cf07-599b-48d9-a6b8-69ec797279b4 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:25.434363 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988cf07-599b-48d9-a6b8-69ec797279b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:25.434640 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988cf07-599b-48d9-a6b8-69ec797279b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:25.443384 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f988cf07-599b-48d9-a6b8-69ec797279b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:25.445367 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f988cf07-599b-48d9-a6b8-69ec797279b4 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:25.454596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1286/2553] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:06:25 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:06:35.196518 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843e6eb7-68c0-4285-95b1-76f6430e3f4c None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:35.256674 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843e6eb7-68c0-4285-95b1-76f6430e3f4c None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:35.261702 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c230b725-c8d9-4ed0-9bda-8a32e35877d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:35.262569 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230b725-c8d9-4ed0-9bda-8a32e35877d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:35.262851 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230b725-c8d9-4ed0-9bda-8a32e35877d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:35.263124 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230b725-c8d9-4ed0-9bda-8a32e35877d9 admin admin] PATH_INFO: `/v3/users/d7047dd629cc4e23a7fc3b2e49b2d60f` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:35.264308 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c230b725-c8d9-4ed0-9bda-8a32e35877d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7047dd629cc4e23a7fc3b2e49b2d60f)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:35.266244 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c230b725-c8d9-4ed0-9bda-8a32e35877d9 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:35.314077 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-c230b725-c8d9-4ed0-9bda-8a32e35877d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7047dd629cc4e23a7fc3b2e49b2d60f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:35.314723 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-c230b725-c8d9-4ed0-9bda-8a32e35877d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7047dd629cc4e23a7fc3b2e49b2d60f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:35.324507 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-c230b725-c8d9-4ed0-9bda-8a32e35877d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7047dd629cc4e23a7fc3b2e49b2d60f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:35.331463 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-c230b725-c8d9-4ed0-9bda-8a32e35877d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7047dd629cc4e23a7fc3b2e49b2d60f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:35.332091 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-c230b725-c8d9-4ed0-9bda-8a32e35877d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7047dd629cc4e23a7fc3b2e49b2d60f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:35.353581 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1268/2554] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:06:35 2022] DELETE /identity/v3/users/d7047dd629cc4e23a7fc3b2e49b2d60f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:06:35.358010 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a2c4d-8e6c-4c5f-8b4f-4040a9e4533d None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:35.358504 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a2c4d-8e6c-4c5f-8b4f-4040a9e4533d None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:35.358613 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a2c4d-8e6c-4c5f-8b4f-4040a9e4533d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:35.375807 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-9e1a2c4d-8e6c-4c5f-8b4f-4040a9e4533d None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:06:35.377024 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-9e1a2c4d-8e6c-4c5f-8b4f-4040a9e4533d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:35.444162 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1287/2555] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:06:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:06:35.454629 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c230b725-c8d9-4ed0-9bda-8a32e35877d9 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:35.456025 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-c230b725-c8d9-4ed0-9bda-8a32e35877d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:35.512150 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c230b725-c8d9-4ed0-9bda-8a32e35877d9 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:35.515766 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e932105-c88c-4f58-8bb9-7117b096f491 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:35.516404 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e932105-c88c-4f58-8bb9-7117b096f491 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:35.516713 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e932105-c88c-4f58-8bb9-7117b096f491 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:35.516986 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e932105-c88c-4f58-8bb9-7117b096f491 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:35.526442 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e932105-c88c-4f58-8bb9-7117b096f491 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:35.528115 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e932105-c88c-4f58-8bb9-7117b096f491 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:35.537255 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1269/2556] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:06:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:06:35.797017 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a2c4d-8e6c-4c5f-8b4f-4040a9e4533d None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:35.798476 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-9e1a2c4d-8e6c-4c5f-8b4f-4040a9e4533d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:35.810686 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a2c4d-8e6c-4c5f-8b4f-4040a9e4533d None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:35.815501 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4973ae4-e1f3-44f7-8153-fe17bb196617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:35.816286 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4973ae4-e1f3-44f7-8153-fe17bb196617 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:35.816566 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4973ae4-e1f3-44f7-8153-fe17bb196617 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:35.816879 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4973ae4-e1f3-44f7-8153-fe17bb196617 admin admin] PATH_INFO: `/v3/projects/ea4d04cd80ad465a8aa19d5ac5bcab55` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:35.818180 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4973ae4-e1f3-44f7-8153-fe17bb196617 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea4d04cd80ad465a8aa19d5ac5bcab55)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:35.820179 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4973ae4-e1f3-44f7-8153-fe17bb196617 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:35.868303 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f4973ae4-e1f3-44f7-8153-fe17bb196617 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea4d04cd80ad465a8aa19d5ac5bcab55', 'request_id': 'req-f4973ae4-e1f3-44f7-8153-fe17bb196617'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:35.873587 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f4973ae4-e1f3-44f7-8153-fe17bb196617 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea4d04cd80ad465a8aa19d5ac5bcab55', 'request_id': 'req-f4973ae4-e1f3-44f7-8153-fe17bb196617'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:35.880487 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f4973ae4-e1f3-44f7-8153-fe17bb196617 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea4d04cd80ad465a8aa19d5ac5bcab55', 'request_id': 'req-f4973ae4-e1f3-44f7-8153-fe17bb196617'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:35.892547 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f4973ae4-e1f3-44f7-8153-fe17bb196617 admin admin] The token cache is being invalidate because project ea4d04cd80ad465a8aa19d5ac5bcab55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:06:35.892860 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f4973ae4-e1f3-44f7-8153-fe17bb196617 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:35.893500 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f4973ae4-e1f3-44f7-8153-fe17bb196617 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:06:35.895047 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1288/2557] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:06:35 2022] DELETE /identity/v3/projects/ea4d04cd80ad465a8aa19d5ac5bcab55 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:06:55.286674 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb86086-34bf-41c8-a897-e836c9ee54f0 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:55.287517 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb86086-34bf-41c8-a897-e836c9ee54f0 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:55.287698 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb86086-34bf-41c8-a897-e836c9ee54f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:55.306268 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-efb86086-34bf-41c8-a897-e836c9ee54f0 None None] MFA Rules not processed for user `a7dc1bc53c59423e934f7831ea022c73`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:06:55.307498 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-efb86086-34bf-41c8-a897-e836c9ee54f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:55.362018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1270/2558] 10.222.0.23 () {58 vars in 1009 bytes} [Fri May 6 15:06:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:06:55.375294 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4973ae4-e1f3-44f7-8153-fe17bb196617 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:06:55.376891 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f4973ae4-e1f3-44f7-8153-fe17bb196617 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:06:55.452738 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4973ae4-e1f3-44f7-8153-fe17bb196617 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:06:55.457881 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed838ac9-fe63-4380-bc14-3525f2e8ace3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf9eed62c8f1409282c2af6488426a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:06:55.458707 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed838ac9-fe63-4380-bc14-3525f2e8ace3 service swift] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:06:55.459001 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed838ac9-fe63-4380-bc14-3525f2e8ace3 service swift] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:06:55.459283 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed838ac9-fe63-4380-bc14-3525f2e8ace3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:06:55.470092 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed838ac9-fe63-4380-bc14-3525f2e8ace3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:06:55.471975 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed838ac9-fe63-4380-bc14-3525f2e8ace3 service swift] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:06:55.481945 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1289/2559] 10.222.0.23 () {60 vars in 1384 bytes} [Fri May 6 15:06:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 06 15:07:09.404127 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb86086-34bf-41c8-a897-e836c9ee54f0 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:09.406354 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-efb86086-34bf-41c8-a897-e836c9ee54f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:09.464880 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb86086-34bf-41c8-a897-e836c9ee54f0 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:09.469809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1039228e-fbd8-47e1-b5c5-3db99f767d67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:09.470499 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1039228e-fbd8-47e1-b5c5-3db99f767d67 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:09.470990 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1039228e-fbd8-47e1-b5c5-3db99f767d67 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:09.471156 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1039228e-fbd8-47e1-b5c5-3db99f767d67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:09.473209 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-1039228e-fbd8-47e1-b5c5-3db99f767d67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:09.536313 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1039228e-fbd8-47e1-b5c5-3db99f767d67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:09.537778 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1039228e-fbd8-47e1-b5c5-3db99f767d67 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:09.548904 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1271/2560] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:07:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:07:09.852256 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71526917-cad7-4191-b180-3ec11fb7c73e None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:09.852894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71526917-cad7-4191-b180-3ec11fb7c73e None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:09.853007 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71526917-cad7-4191-b180-3ec11fb7c73e None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:09.854439 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1290/2561] 10.222.0.23 () {54 vars in 921 bytes} [Fri May 6 15:07:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 15:07:09.861740 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de46d59f-7187-4ef6-8701-fdfb5414f64f None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:09.861968 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de46d59f-7187-4ef6-8701-fdfb5414f64f None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:09.862158 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de46d59f-7187-4ef6-8701-fdfb5414f64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:09.883403 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-de46d59f-7187-4ef6-8701-fdfb5414f64f None None] MFA Rules not processed for user `fc8cf09642ec4e10a80badcf8b585ef8`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:07:09.884657 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-de46d59f-7187-4ef6-8701-fdfb5414f64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:09.956300 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1272/2562] 10.222.0.23 () {58 vars in 1018 bytes} [Fri May 6 15:07:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:07:09.968850 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71526917-cad7-4191-b180-3ec11fb7c73e None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:09.970618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-71526917-cad7-4191-b180-3ec11fb7c73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:10.029068 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71526917-cad7-4191-b180-3ec11fb7c73e None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:10.033769 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a5857-e3b5-4be6-997f-1576ccd83eb8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9be93f9ff7e46babc8ee87b50b9c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:10.034418 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a5857-e3b5-4be6-997f-1576ccd83eb8 service glance] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:10.034704 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a5857-e3b5-4be6-997f-1576ccd83eb8 service glance] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:10.034997 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a5857-e3b5-4be6-997f-1576ccd83eb8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:10.046767 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a5857-e3b5-4be6-997f-1576ccd83eb8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:10.047614 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a5857-e3b5-4be6-997f-1576ccd83eb8 service glance] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:10.059069 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1291/2563] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:07:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 15:07:10.128160 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c2b5d-e075-4f88-b0a1-5a90ad354e26 None None] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:10.128446 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c2b5d-e075-4f88-b0a1-5a90ad354e26 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:10.128760 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c2b5d-e075-4f88-b0a1-5a90ad354e26 None None] PATH_INFO: `/` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:10.130130 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1273/2564] 10.222.0.23 () {54 vars in 921 bytes} [Fri May 6 15:07:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 15:07:10.135220 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558727b-1a6c-4571-9bea-d09dc443159f None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:10.135645 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558727b-1a6c-4571-9bea-d09dc443159f None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:10.136117 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558727b-1a6c-4571-9bea-d09dc443159f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:10.155888 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-1558727b-1a6c-4571-9bea-d09dc443159f None None] MFA Rules not processed for user `4dcd1542d8f643c4996142a88b9b6abb`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:07:10.157440 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-1558727b-1a6c-4571-9bea-d09dc443159f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:10.164527 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1292/2565] 10.222.0.23 () {58 vars in 1018 bytes} [Fri May 6 15:07:10 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 15:07:10.177559 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c2b5d-e075-4f88-b0a1-5a90ad354e26 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:10.179725 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-467c2b5d-e075-4f88-b0a1-5a90ad354e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:10.193381 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c2b5d-e075-4f88-b0a1-5a90ad354e26 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:10.197099 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f935c180-acd1-4865-81d7-1b6abb0ec26e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:10.197746 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935c180-acd1-4865-81d7-1b6abb0ec26e service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:10.198029 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935c180-acd1-4865-81d7-1b6abb0ec26e service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:10.198311 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935c180-acd1-4865-81d7-1b6abb0ec26e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:10.205684 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f935c180-acd1-4865-81d7-1b6abb0ec26e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:10.207158 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f935c180-acd1-4865-81d7-1b6abb0ec26e service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:10.215169 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1274/2566] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:07:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:07:10.301271 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d5ca2-12ec-412a-ab19-73a81faa5ac1 None None] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:10.301558 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d5ca2-12ec-412a-ab19-73a81faa5ac1 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:10.301843 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d5ca2-12ec-412a-ab19-73a81faa5ac1 None None] PATH_INFO: `/` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:10.303249 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1293/2567] 10.222.0.23 () {54 vars in 921 bytes} [Fri May 6 15:07:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 06 15:07:10.308849 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96fca67-b217-41c4-a87a-e0e60079a1ad None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:10.309143 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96fca67-b217-41c4-a87a-e0e60079a1ad None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:10.309427 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96fca67-b217-41c4-a87a-e0e60079a1ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:10.328228 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-b96fca67-b217-41c4-a87a-e0e60079a1ad None None] MFA Rules not processed for user `fc8cf09642ec4e10a80badcf8b585ef8`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:07:10.329469 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b96fca67-b217-41c4-a87a-e0e60079a1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:10.339130 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1275/2568] 10.222.0.23 () {58 vars in 1018 bytes} [Fri May 6 15:07:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:07:10.350457 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d5ca2-12ec-412a-ab19-73a81faa5ac1 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:10.362419 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d5ca2-12ec-412a-ab19-73a81faa5ac1 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:10.366209 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc076e1-2fe2-4551-9809-2af0f808647a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:10.366854 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc076e1-2fe2-4551-9809-2af0f808647a service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:10.367169 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc076e1-2fe2-4551-9809-2af0f808647a service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:10.367447 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc076e1-2fe2-4551-9809-2af0f808647a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:10.377972 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc076e1-2fe2-4551-9809-2af0f808647a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:10.379399 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc076e1-2fe2-4551-9809-2af0f808647a service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:10.390370 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1294/2569] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:07:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:07:12.183296 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd9e14-8889-4c69-a36d-db3b35e60d6d None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:12.183769 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd9e14-8889-4c69-a36d-db3b35e60d6d None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:12.183879 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd9e14-8889-4c69-a36d-db3b35e60d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:12.202717 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-29cd9e14-8889-4c69-a36d-db3b35e60d6d None None] MFA Rules not processed for user `a7dc1bc53c59423e934f7831ea022c73`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:07:12.204012 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-29cd9e14-8889-4c69-a36d-db3b35e60d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:12.212126 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1276/2570] 10.222.0.23 () {58 vars in 1009 bytes} [Fri May 6 15:07:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:07:12.224762 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc076e1-2fe2-4551-9809-2af0f808647a service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:12.237358 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc076e1-2fe2-4551-9809-2af0f808647a service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:12.242881 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce05e2-bcce-4db6-8ec9-fba0e8b555d1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf9eed62c8f1409282c2af6488426a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:12.243537 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce05e2-bcce-4db6-8ec9-fba0e8b555d1 service swift] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:12.243823 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce05e2-bcce-4db6-8ec9-fba0e8b555d1 service swift] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:12.244100 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce05e2-bcce-4db6-8ec9-fba0e8b555d1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:12.252764 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccce05e2-bcce-4db6-8ec9-fba0e8b555d1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:12.254345 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccce05e2-bcce-4db6-8ec9-fba0e8b555d1 service swift] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:12.271401 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1295/2571] 10.222.0.23 () {60 vars in 1384 bytes} [Fri May 6 15:07:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 06 15:07:13.402765 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cd9e14-8889-4c69-a36d-db3b35e60d6d None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:13.404316 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-29cd9e14-8889-4c69-a36d-db3b35e60d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:13.426455 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cd9e14-8889-4c69-a36d-db3b35e60d6d None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:13.445300 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97826b18-4d14-469d-b64b-9cb72aff378d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:13.446210 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97826b18-4d14-469d-b64b-9cb72aff378d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:13.446853 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97826b18-4d14-469d-b64b-9cb72aff378d admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:13.447302 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97826b18-4d14-469d-b64b-9cb72aff378d admin admin] PATH_INFO: `/v3/users/afd9f5519134415b8fc8a8c08149c502` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:13.449406 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97826b18-4d14-469d-b64b-9cb72aff378d admin admin] RBAC: Authorizing `identity:delete_user(user_id=afd9f5519134415b8fc8a8c08149c502)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:13.457449 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97826b18-4d14-469d-b64b-9cb72aff378d admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:13.557790 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-97826b18-4d14-469d-b64b-9cb72aff378d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afd9f5519134415b8fc8a8c08149c502'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:13.558739 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-97826b18-4d14-469d-b64b-9cb72aff378d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afd9f5519134415b8fc8a8c08149c502'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:13.571389 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-97826b18-4d14-469d-b64b-9cb72aff378d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afd9f5519134415b8fc8a8c08149c502'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:13.582456 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-97826b18-4d14-469d-b64b-9cb72aff378d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afd9f5519134415b8fc8a8c08149c502'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:13.583411 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-97826b18-4d14-469d-b64b-9cb72aff378d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afd9f5519134415b8fc8a8c08149c502'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:13.607962 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1277/2572] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:07:13 2022] DELETE /identity/v3/users/afd9f5519134415b8fc8a8c08149c502 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:07:13.917261 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce05e2-bcce-4db6-8ec9-fba0e8b555d1 service swift] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:13.938227 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce05e2-bcce-4db6-8ec9-fba0e8b555d1 service swift] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:14.021869 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc38653-ac07-491a-bb4e-074d20a45f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:14.022922 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc38653-ac07-491a-bb4e-074d20a45f99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:14.023310 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc38653-ac07-491a-bb4e-074d20a45f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:14.023816 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc38653-ac07-491a-bb4e-074d20a45f99 admin admin] PATH_INFO: `/v3/projects/5ad13dae3f6649fb8d8d2bdd1f1caaef` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:14.025732 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc38653-ac07-491a-bb4e-074d20a45f99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ad13dae3f6649fb8d8d2bdd1f1caaef)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:14.028064 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc38653-ac07-491a-bb4e-074d20a45f99 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:14.091435 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ebc38653-ac07-491a-bb4e-074d20a45f99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ad13dae3f6649fb8d8d2bdd1f1caaef', 'request_id': 'req-ebc38653-ac07-491a-bb4e-074d20a45f99'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:14.099295 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ebc38653-ac07-491a-bb4e-074d20a45f99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ad13dae3f6649fb8d8d2bdd1f1caaef', 'request_id': 'req-ebc38653-ac07-491a-bb4e-074d20a45f99'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:14.109453 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ebc38653-ac07-491a-bb4e-074d20a45f99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ad13dae3f6649fb8d8d2bdd1f1caaef', 'request_id': 'req-ebc38653-ac07-491a-bb4e-074d20a45f99'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:14.125899 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ebc38653-ac07-491a-bb4e-074d20a45f99 admin admin] The token cache is being invalidate because project 5ad13dae3f6649fb8d8d2bdd1f1caaef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:07:14.126456 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ebc38653-ac07-491a-bb4e-074d20a45f99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:14.127369 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-ebc38653-ac07-491a-bb4e-074d20a45f99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:14.135260 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457a929-1020-4591-b0eb-69d6836309dd None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:14.135552 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457a929-1020-4591-b0eb-69d6836309dd None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:14.135831 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457a929-1020-4591-b0eb-69d6836309dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:14.136703 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1296/2573] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:07:13 2022] DELETE /identity/v3/projects/5ad13dae3f6649fb8d8d2bdd1f1caaef => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:07:14.174186 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-d457a929-1020-4591-b0eb-69d6836309dd None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:07:14.175749 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d457a929-1020-4591-b0eb-69d6836309dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:14.251224 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1278/2574] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:07:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 15:07:14.258637 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae16ef3-1ea5-4e85-9fdc-187341a7654b None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:14.258926 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae16ef3-1ea5-4e85-9fdc-187341a7654b None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:14.259206 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae16ef3-1ea5-4e85-9fdc-187341a7654b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:14.280347 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-2ae16ef3-1ea5-4e85-9fdc-187341a7654b None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:07:14.281723 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-2ae16ef3-1ea5-4e85-9fdc-187341a7654b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:14.293801 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1297/2575] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:07:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:07:14.297861 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d457a929-1020-4591-b0eb-69d6836309dd None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:14.307826 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d457a929-1020-4591-b0eb-69d6836309dd None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:14.312540 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b0af2-2dd1-4a95-8ac5-aef1ed4272fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:14.313284 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b0af2-2dd1-4a95-8ac5-aef1ed4272fd admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:14.313569 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b0af2-2dd1-4a95-8ac5-aef1ed4272fd admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:14.313901 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b0af2-2dd1-4a95-8ac5-aef1ed4272fd admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:14.314617 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5b0af2-2dd1-4a95-8ac5-aef1ed4272fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 15:07:14.314925 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5b0af2-2dd1-4a95-8ac5-aef1ed4272fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:14.316561 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5b0af2-2dd1-4a95-8ac5-aef1ed4272fd admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:14.333071 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1279/2576] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 15:07:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 15:07:14.335463 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae16ef3-1ea5-4e85-9fdc-187341a7654b None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:14.345798 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae16ef3-1ea5-4e85-9fdc-187341a7654b None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:14.350708 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99862a13-cc65-4c48-bcfb-d94967815ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:14.351775 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99862a13-cc65-4c48-bcfb-d94967815ffb admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:14.352240 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99862a13-cc65-4c48-bcfb-d94967815ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:14.352717 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99862a13-cc65-4c48-bcfb-d94967815ffb admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:14.353828 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99862a13-cc65-4c48-bcfb-d94967815ffb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:14.355942 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99862a13-cc65-4c48-bcfb-d94967815ffb admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:14.389971 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1298/2577] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:07:14 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:07:14.390659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b0af2-2dd1-4a95-8ac5-aef1ed4272fd admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:14.401108 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b0af2-2dd1-4a95-8ac5-aef1ed4272fd admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:14.476818 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c550bc-f676-43e7-95f2-bf846d7f0d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:14.477524 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c550bc-f676-43e7-95f2-bf846d7f0d66 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:14.477765 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c550bc-f676-43e7-95f2-bf846d7f0d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:14.478038 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c550bc-f676-43e7-95f2-bf846d7f0d66 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:14.478730 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c550bc-f676-43e7-95f2-bf846d7f0d66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:14.480676 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c550bc-f676-43e7-95f2-bf846d7f0d66 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:14.506971 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1280/2578] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:07:14 2022] POST /identity/v3/users => generated 287 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 15:07:14.510365 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99862a13-cc65-4c48-bcfb-d94967815ffb admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:14.521614 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99862a13-cc65-4c48-bcfb-d94967815ffb admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:14.527224 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fef152b-2046-45b7-9d8a-569566b2b1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:14.527936 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef152b-2046-45b7-9d8a-569566b2b1a0 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:14.528218 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef152b-2046-45b7-9d8a-569566b2b1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:14.528534 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef152b-2046-45b7-9d8a-569566b2b1a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:14.529313 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fef152b-2046-45b7-9d8a-569566b2b1a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:14.532152 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fef152b-2046-45b7-9d8a-569566b2b1a0 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:14.544673 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1299/2579] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:07:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:07:14.548671 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c550bc-f676-43e7-95f2-bf846d7f0d66 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:14.559935 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c550bc-f676-43e7-95f2-bf846d7f0d66 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:14.565751 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be97b351-49df-4e0c-9158-2344c5388784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:14.566428 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97b351-49df-4e0c-9158-2344c5388784 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:14.566782 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97b351-49df-4e0c-9158-2344c5388784 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:14.567204 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97b351-49df-4e0c-9158-2344c5388784 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:14.568015 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be97b351-49df-4e0c-9158-2344c5388784 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:14.569659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be97b351-49df-4e0c-9158-2344c5388784 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:14.584369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1281/2580] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:07:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 06 15:07:14.586402 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fef152b-2046-45b7-9d8a-569566b2b1a0 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:14.596981 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fef152b-2046-45b7-9d8a-569566b2b1a0 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:14.601987 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d737ec9a-d9c1-4097-9e04-4d75e7bc67e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:14.602889 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737ec9a-d9c1-4097-9e04-4d75e7bc67e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:14.603204 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737ec9a-d9c1-4097-9e04-4d75e7bc67e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:14.603481 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737ec9a-d9c1-4097-9e04-4d75e7bc67e9 admin admin] PATH_INFO: `/v3/projects/d533d419d73d44f6adfd83a8ad30dbe0/users/85ff235b28ce4885ac84ee3471e2f0b0/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:14.619391 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d737ec9a-d9c1-4097-9e04-4d75e7bc67e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d533d419d73d44f6adfd83a8ad30dbe0, user_id=85ff235b28ce4885ac84ee3471e2f0b0, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:14.622374 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d737ec9a-d9c1-4097-9e04-4d75e7bc67e9 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:14.638030 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1300/2581] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:07:14 2022] PUT /identity/v3/projects/d533d419d73d44f6adfd83a8ad30dbe0/users/85ff235b28ce4885ac84ee3471e2f0b0/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:07:14.649257 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc69f3-9ac1-43e7-b5ce-a1e745f30561 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:14.649511 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc69f3-9ac1-43e7-b5ce-a1e745f30561 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:14.649726 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc69f3-9ac1-43e7-b5ce-a1e745f30561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:14.682721 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-a1fc69f3-9ac1-43e7-b5ce-a1e745f30561 None None] MFA Rules not processed for user `85ff235b28ce4885ac84ee3471e2f0b0`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:07:14.684181 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-a1fc69f3-9ac1-43e7-b5ce-a1e745f30561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:14.765252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1282/2582] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:07:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:07:14.776517 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d737ec9a-d9c1-4097-9e04-4d75e7bc67e9 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:14.778584 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-d737ec9a-d9c1-4097-9e04-4d75e7bc67e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:14.865056 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d737ec9a-d9c1-4097-9e04-4d75e7bc67e9 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:14.870833 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4189d8-6831-4168-bef5-d4c5b4f06da9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:14.871502 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4189d8-6831-4168-bef5-d4c5b4f06da9 service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:14.871794 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4189d8-6831-4168-bef5-d4c5b4f06da9 service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:14.872069 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4189d8-6831-4168-bef5-d4c5b4f06da9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:14.881841 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4189d8-6831-4168-bef5-d4c5b4f06da9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:14.883583 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4189d8-6831-4168-bef5-d4c5b4f06da9 service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:14.894711 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1301/2583] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:07:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:07:17.703880 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc69f3-9ac1-43e7-b5ce-a1e745f30561 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:17.769429 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc69f3-9ac1-43e7-b5ce-a1e745f30561 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:17.774611 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41239ca6-6644-44b7-8825-8cc865945443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:17.775496 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41239ca6-6644-44b7-8825-8cc865945443 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:17.775780 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41239ca6-6644-44b7-8825-8cc865945443 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:17.776057 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41239ca6-6644-44b7-8825-8cc865945443 admin admin] PATH_INFO: `/v3/users/85ff235b28ce4885ac84ee3471e2f0b0` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:17.777547 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41239ca6-6644-44b7-8825-8cc865945443 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85ff235b28ce4885ac84ee3471e2f0b0)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:17.780082 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41239ca6-6644-44b7-8825-8cc865945443 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:17.840366 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-41239ca6-6644-44b7-8825-8cc865945443 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85ff235b28ce4885ac84ee3471e2f0b0'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:17.841041 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-41239ca6-6644-44b7-8825-8cc865945443 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85ff235b28ce4885ac84ee3471e2f0b0'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:17.848815 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-41239ca6-6644-44b7-8825-8cc865945443 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85ff235b28ce4885ac84ee3471e2f0b0'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:17.856762 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-41239ca6-6644-44b7-8825-8cc865945443 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85ff235b28ce4885ac84ee3471e2f0b0'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:17.857523 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-41239ca6-6644-44b7-8825-8cc865945443 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85ff235b28ce4885ac84ee3471e2f0b0'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:17.877315 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1283/2584] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:07:17 2022] DELETE /identity/v3/users/85ff235b28ce4885ac84ee3471e2f0b0 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:07:17.881651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff47ec-713e-48b3-84fc-a8b33c8ad2e1 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:17.882321 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff47ec-713e-48b3-84fc-a8b33c8ad2e1 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:17.882321 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff47ec-713e-48b3-84fc-a8b33c8ad2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:17.899657 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-ebff47ec-713e-48b3-84fc-a8b33c8ad2e1 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:07:17.900921 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-ebff47ec-713e-48b3-84fc-a8b33c8ad2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:17.962982 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1302/2585] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:07:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:07:17.974561 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41239ca6-6644-44b7-8825-8cc865945443 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:17.975982 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-41239ca6-6644-44b7-8825-8cc865945443 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:18.040146 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41239ca6-6644-44b7-8825-8cc865945443 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:18.043980 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360879cc-f8e7-4a70-bea0-2e8031bdec95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:18.044665 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360879cc-f8e7-4a70-bea0-2e8031bdec95 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:18.044953 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360879cc-f8e7-4a70-bea0-2e8031bdec95 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:18.045263 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360879cc-f8e7-4a70-bea0-2e8031bdec95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:18.055156 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360879cc-f8e7-4a70-bea0-2e8031bdec95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:18.058909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360879cc-f8e7-4a70-bea0-2e8031bdec95 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:18.071028 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1284/2586] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:07:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:07:18.375975 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebff47ec-713e-48b3-84fc-a8b33c8ad2e1 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:18.377450 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-ebff47ec-713e-48b3-84fc-a8b33c8ad2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:18.390263 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebff47ec-713e-48b3-84fc-a8b33c8ad2e1 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:18.395069 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f17b9f0-4c7f-4178-a66b-a28ba6a4fe6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:18.395952 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17b9f0-4c7f-4178-a66b-a28ba6a4fe6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:18.396235 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17b9f0-4c7f-4178-a66b-a28ba6a4fe6e admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:18.396514 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17b9f0-4c7f-4178-a66b-a28ba6a4fe6e admin admin] PATH_INFO: `/v3/projects/d533d419d73d44f6adfd83a8ad30dbe0` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:18.397768 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f17b9f0-4c7f-4178-a66b-a28ba6a4fe6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d533d419d73d44f6adfd83a8ad30dbe0)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:18.399715 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f17b9f0-4c7f-4178-a66b-a28ba6a4fe6e admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:18.451601 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-6f17b9f0-4c7f-4178-a66b-a28ba6a4fe6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd533d419d73d44f6adfd83a8ad30dbe0', 'request_id': 'req-6f17b9f0-4c7f-4178-a66b-a28ba6a4fe6e'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:18.457349 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-6f17b9f0-4c7f-4178-a66b-a28ba6a4fe6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd533d419d73d44f6adfd83a8ad30dbe0', 'request_id': 'req-6f17b9f0-4c7f-4178-a66b-a28ba6a4fe6e'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:18.473876 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-6f17b9f0-4c7f-4178-a66b-a28ba6a4fe6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd533d419d73d44f6adfd83a8ad30dbe0', 'request_id': 'req-6f17b9f0-4c7f-4178-a66b-a28ba6a4fe6e'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:18.486617 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-6f17b9f0-4c7f-4178-a66b-a28ba6a4fe6e admin admin] The token cache is being invalidate because project d533d419d73d44f6adfd83a8ad30dbe0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:07:18.486911 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-6f17b9f0-4c7f-4178-a66b-a28ba6a4fe6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:18.487568 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-6f17b9f0-4c7f-4178-a66b-a28ba6a4fe6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:18.489159 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1303/2587] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:07:18 2022] DELETE /identity/v3/projects/d533d419d73d44f6adfd83a8ad30dbe0 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:07:18.494313 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a07eb4-ecfd-4e2b-83d6-adc24c331792 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:18.494636 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a07eb4-ecfd-4e2b-83d6-adc24c331792 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:18.494936 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a07eb4-ecfd-4e2b-83d6-adc24c331792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:18.523248 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-e5a07eb4-ecfd-4e2b-83d6-adc24c331792 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:07:18.525568 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-e5a07eb4-ecfd-4e2b-83d6-adc24c331792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:18.591136 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1285/2588] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:07:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:07:18.600147 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2903c18-d4c7-4bd1-a025-8646ee078b39 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:18.600515 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2903c18-d4c7-4bd1-a025-8646ee078b39 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:18.600903 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2903c18-d4c7-4bd1-a025-8646ee078b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:18.620419 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-e2903c18-d4c7-4bd1-a025-8646ee078b39 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:07:18.622402 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e2903c18-d4c7-4bd1-a025-8646ee078b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:18.642237 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a07eb4-ecfd-4e2b-83d6-adc24c331792 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:18.653123 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1304/2589] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:07:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:07:18.653617 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a07eb4-ecfd-4e2b-83d6-adc24c331792 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:18.660160 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8067a0d8-02ad-42b2-8423-b482dae8772c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:18.660869 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8067a0d8-02ad-42b2-8423-b482dae8772c admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:18.661149 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8067a0d8-02ad-42b2-8423-b482dae8772c admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:18.661424 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8067a0d8-02ad-42b2-8423-b482dae8772c admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:18.662164 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8067a0d8-02ad-42b2-8423-b482dae8772c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 15:07:18.662467 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8067a0d8-02ad-42b2-8423-b482dae8772c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:18.664097 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8067a0d8-02ad-42b2-8423-b482dae8772c admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:18.681037 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1286/2590] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 15:07:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 15:07:18.683677 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2903c18-d4c7-4bd1-a025-8646ee078b39 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:18.693730 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2903c18-d4c7-4bd1-a025-8646ee078b39 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:18.698367 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c793b4fa-cab5-4112-a0af-453bc946a16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:18.699026 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793b4fa-cab5-4112-a0af-453bc946a16f admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:18.699307 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793b4fa-cab5-4112-a0af-453bc946a16f admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:18.699580 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793b4fa-cab5-4112-a0af-453bc946a16f admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:18.700252 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c793b4fa-cab5-4112-a0af-453bc946a16f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:18.703188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c793b4fa-cab5-4112-a0af-453bc946a16f admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:18.730321 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1305/2591] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:07:18 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 15:07:18.735091 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8067a0d8-02ad-42b2-8423-b482dae8772c admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:18.748133 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8067a0d8-02ad-42b2-8423-b482dae8772c admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:18.822161 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf3620-5a12-42ca-ac10-5c2b33b57946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:18.822833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf3620-5a12-42ca-ac10-5c2b33b57946 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:18.823113 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf3620-5a12-42ca-ac10-5c2b33b57946 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:18.823384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf3620-5a12-42ca-ac10-5c2b33b57946 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:18.824067 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cf3620-5a12-42ca-ac10-5c2b33b57946 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:18.826039 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cf3620-5a12-42ca-ac10-5c2b33b57946 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:18.851138 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1287/2592] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:07:18 2022] POST /identity/v3/users => generated 283 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:07:18.855203 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c793b4fa-cab5-4112-a0af-453bc946a16f admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:18.865684 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c793b4fa-cab5-4112-a0af-453bc946a16f admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:18.870384 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770a0c7d-bfdf-48e0-adcf-4b68229fa3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:18.871041 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a0c7d-bfdf-48e0-adcf-4b68229fa3bf admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:18.871325 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a0c7d-bfdf-48e0-adcf-4b68229fa3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:18.871600 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a0c7d-bfdf-48e0-adcf-4b68229fa3bf admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:18.872313 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770a0c7d-bfdf-48e0-adcf-4b68229fa3bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:18.874015 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770a0c7d-bfdf-48e0-adcf-4b68229fa3bf admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:18.884197 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1306/2593] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:07:18.888477 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf3620-5a12-42ca-ac10-5c2b33b57946 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:18.900592 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf3620-5a12-42ca-ac10-5c2b33b57946 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:18.906430 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab12c7-9b75-487b-8ac5-b71fa332924d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:18.907099 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab12c7-9b75-487b-8ac5-b71fa332924d admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:18.907380 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab12c7-9b75-487b-8ac5-b71fa332924d admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:18.907654 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab12c7-9b75-487b-8ac5-b71fa332924d admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:18.908383 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faab12c7-9b75-487b-8ac5-b71fa332924d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:18.910027 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faab12c7-9b75-487b-8ac5-b71fa332924d admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:18.921661 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1288/2594] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:07:18.925554 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770a0c7d-bfdf-48e0-adcf-4b68229fa3bf admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:18.936901 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770a0c7d-bfdf-48e0-adcf-4b68229fa3bf admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:18.942081 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef7fc2e-61a9-4e84-902e-3669a5eafb3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:18.943030 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef7fc2e-61a9-4e84-902e-3669a5eafb3d admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:18.943319 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef7fc2e-61a9-4e84-902e-3669a5eafb3d admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:18.943676 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef7fc2e-61a9-4e84-902e-3669a5eafb3d admin admin] PATH_INFO: `/v3/projects/5e3b290a4b66445d8fb970821c12ce0f/users/5423db71126847609bb438d72b6351fa/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:18.960015 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef7fc2e-61a9-4e84-902e-3669a5eafb3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e3b290a4b66445d8fb970821c12ce0f, user_id=5423db71126847609bb438d72b6351fa, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:18.962811 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef7fc2e-61a9-4e84-902e-3669a5eafb3d admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:18.978063 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1307/2595] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:07:18 2022] PUT /identity/v3/projects/5e3b290a4b66445d8fb970821c12ce0f/users/5423db71126847609bb438d72b6351fa/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:07:18.989113 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477f17b-887a-440e-894a-348cb7f5f025 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:18.989374 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477f17b-887a-440e-894a-348cb7f5f025 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:18.989654 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477f17b-887a-440e-894a-348cb7f5f025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:19.019888 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-a477f17b-887a-440e-894a-348cb7f5f025 None None] MFA Rules not processed for user `5423db71126847609bb438d72b6351fa`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:07:19.021156 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-a477f17b-887a-440e-894a-348cb7f5f025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:19.094231 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1289/2596] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:07:18 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:07:19.099348 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef7fc2e-61a9-4e84-902e-3669a5eafb3d admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:19.180222 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef7fc2e-61a9-4e84-902e-3669a5eafb3d admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:19.185179 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae25d0a-859f-46eb-9932-19cf7b0bdd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:19.185891 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae25d0a-859f-46eb-9932-19cf7b0bdd10 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:19.186675 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae25d0a-859f-46eb-9932-19cf7b0bdd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:19.186675 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae25d0a-859f-46eb-9932-19cf7b0bdd10 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:19.187294 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae25d0a-859f-46eb-9932-19cf7b0bdd10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:19.189423 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae25d0a-859f-46eb-9932-19cf7b0bdd10 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:19.220868 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1308/2597] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:07:19 2022] POST /identity/v3/projects => generated 365 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:07:19.235852 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a477f17b-887a-440e-894a-348cb7f5f025 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:19.250368 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a477f17b-887a-440e-894a-348cb7f5f025 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:19.322803 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad71d0e-9364-45eb-8d52-8c04d30df19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:19.323503 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad71d0e-9364-45eb-8d52-8c04d30df19e admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:19.323851 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad71d0e-9364-45eb-8d52-8c04d30df19e admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:19.324188 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad71d0e-9364-45eb-8d52-8c04d30df19e admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:19.324986 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad71d0e-9364-45eb-8d52-8c04d30df19e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:19.327744 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad71d0e-9364-45eb-8d52-8c04d30df19e admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:19.349951 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1290/2598] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:07:19 2022] POST /identity/v3/users => generated 289 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 15:07:19.354075 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae25d0a-859f-46eb-9932-19cf7b0bdd10 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:19.364839 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae25d0a-859f-46eb-9932-19cf7b0bdd10 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:19.369797 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c2607-0578-4a09-98b1-0d475d7eab11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:19.370486 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c2607-0578-4a09-98b1-0d475d7eab11 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:19.370768 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c2607-0578-4a09-98b1-0d475d7eab11 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:19.371047 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c2607-0578-4a09-98b1-0d475d7eab11 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:19.371931 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c2607-0578-4a09-98b1-0d475d7eab11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:19.374345 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c2607-0578-4a09-98b1-0d475d7eab11 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:19.386346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1309/2599] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:07:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:07:19.391529 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad71d0e-9364-45eb-8d52-8c04d30df19e admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:19.402269 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad71d0e-9364-45eb-8d52-8c04d30df19e admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:19.407348 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3a6cfc-358a-490a-ad1d-94f84152b0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:19.408249 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a6cfc-358a-490a-ad1d-94f84152b0e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:19.408532 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a6cfc-358a-490a-ad1d-94f84152b0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:19.408842 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a6cfc-358a-490a-ad1d-94f84152b0e4 admin admin] PATH_INFO: `/v3/projects/05c2eec00c124967bf9f9fdbcbc03938/users/7ba2e24bd4484fe4a8edecc32d10d5dd/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:19.426060 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3a6cfc-358a-490a-ad1d-94f84152b0e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05c2eec00c124967bf9f9fdbcbc03938, user_id=7ba2e24bd4484fe4a8edecc32d10d5dd, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:19.428921 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3a6cfc-358a-490a-ad1d-94f84152b0e4 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:19.444054 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1291/2600] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:07:19 2022] PUT /identity/v3/projects/05c2eec00c124967bf9f9fdbcbc03938/users/7ba2e24bd4484fe4a8edecc32d10d5dd/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:07:19.448570 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c2607-0578-4a09-98b1-0d475d7eab11 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:19.460242 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c2607-0578-4a09-98b1-0d475d7eab11 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:19.524316 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c6ebc-ea3c-4f12-b4f3-ddfc0d71947d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:19.525212 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c6ebc-ea3c-4f12-b4f3-ddfc0d71947d admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:19.525568 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c6ebc-ea3c-4f12-b4f3-ddfc0d71947d admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:19.525919 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c6ebc-ea3c-4f12-b4f3-ddfc0d71947d admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:19.526723 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036c6ebc-ea3c-4f12-b4f3-ddfc0d71947d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:19.528746 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036c6ebc-ea3c-4f12-b4f3-ddfc0d71947d admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:19.540796 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1310/2601] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:07:19 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 06 15:07:19.546014 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3a6cfc-358a-490a-ad1d-94f84152b0e4 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:19.556768 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3a6cfc-358a-490a-ad1d-94f84152b0e4 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:19.562627 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f47a514-46fa-4609-adaf-19b8969768bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:19.563677 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47a514-46fa-4609-adaf-19b8969768bd admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:19.564030 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47a514-46fa-4609-adaf-19b8969768bd admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:19.564378 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47a514-46fa-4609-adaf-19b8969768bd admin admin] PATH_INFO: `/v3/projects/05c2eec00c124967bf9f9fdbcbc03938/users/7ba2e24bd4484fe4a8edecc32d10d5dd/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:19.566525 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f47a514-46fa-4609-adaf-19b8969768bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=05c2eec00c124967bf9f9fdbcbc03938, user_id=7ba2e24bd4484fe4a8edecc32d10d5dd, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:19.569410 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f47a514-46fa-4609-adaf-19b8969768bd admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:19.581230 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1292/2602] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:07:19 2022] PUT /identity/v3/projects/05c2eec00c124967bf9f9fdbcbc03938/users/7ba2e24bd4484fe4a8edecc32d10d5dd/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:07:19.593011 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8076da07-22a5-4260-8da0-1892ce67d56d None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:19.593303 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8076da07-22a5-4260-8da0-1892ce67d56d None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:19.593586 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8076da07-22a5-4260-8da0-1892ce67d56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:19.622106 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-8076da07-22a5-4260-8da0-1892ce67d56d None None] MFA Rules not processed for user `7ba2e24bd4484fe4a8edecc32d10d5dd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:07:19.623911 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-8076da07-22a5-4260-8da0-1892ce67d56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:19.710384 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1311/2603] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:07:19 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:07:19.721952 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f47a514-46fa-4609-adaf-19b8969768bd admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:19.723667 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-0f47a514-46fa-4609-adaf-19b8969768bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:19.790426 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f47a514-46fa-4609-adaf-19b8969768bd admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:19.794173 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35477e9a-ec3a-4759-98e7-9b400dd4216b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:19.794820 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35477e9a-ec3a-4759-98e7-9b400dd4216b service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:19.795228 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35477e9a-ec3a-4759-98e7-9b400dd4216b service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:19.795651 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35477e9a-ec3a-4759-98e7-9b400dd4216b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:19.817195 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35477e9a-ec3a-4759-98e7-9b400dd4216b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:19.818816 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35477e9a-ec3a-4759-98e7-9b400dd4216b service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:19.830270 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1293/2604] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:07:19 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:07:19.933430 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8076da07-22a5-4260-8da0-1892ce67d56d None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:19.946679 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8076da07-22a5-4260-8da0-1892ce67d56d None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:19.951217 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee4a24-b7bf-4eb1-9ba3-53ee15b3cd5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:19.952842 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee4a24-b7bf-4eb1-9ba3-53ee15b3cd5b service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:19.953225 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee4a24-b7bf-4eb1-9ba3-53ee15b3cd5b service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:19.953504 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee4a24-b7bf-4eb1-9ba3-53ee15b3cd5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:20.044455 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ee4a24-b7bf-4eb1-9ba3-53ee15b3cd5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:20.046054 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ee4a24-b7bf-4eb1-9ba3-53ee15b3cd5b service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:20.055947 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1312/2605] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:07:19 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:07:30.184182 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81739d-a97d-4d66-a1e6-ad4d12577821 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:30.184734 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81739d-a97d-4d66-a1e6-ad4d12577821 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:30.185037 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81739d-a97d-4d66-a1e6-ad4d12577821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:30.204132 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-fd81739d-a97d-4d66-a1e6-ad4d12577821 None None] MFA Rules not processed for user `a7dc1bc53c59423e934f7831ea022c73`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:07:30.205384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-fd81739d-a97d-4d66-a1e6-ad4d12577821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:30.246947 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee4a24-b7bf-4eb1-9ba3-53ee15b3cd5b service cinder] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:30.266470 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1294/2606] 10.222.0.23 () {58 vars in 967 bytes} [Fri May 6 15:07:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:07:30.293511 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd81739d-a97d-4d66-a1e6-ad4d12577821 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:30.295545 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-fd81739d-a97d-4d66-a1e6-ad4d12577821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:30.319679 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee4a24-b7bf-4eb1-9ba3-53ee15b3cd5b service cinder] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:30.324184 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94548676-43fd-4c16-8e3d-7be41aab4fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:30.325081 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94548676-43fd-4c16-8e3d-7be41aab4fae admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:30.325361 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94548676-43fd-4c16-8e3d-7be41aab4fae admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:30.325628 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94548676-43fd-4c16-8e3d-7be41aab4fae admin admin] PATH_INFO: `/v3/users/5423db71126847609bb438d72b6351fa` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:30.327222 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94548676-43fd-4c16-8e3d-7be41aab4fae admin admin] RBAC: Authorizing `identity:delete_user(user_id=5423db71126847609bb438d72b6351fa)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:30.329195 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94548676-43fd-4c16-8e3d-7be41aab4fae admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:30.372164 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd81739d-a97d-4d66-a1e6-ad4d12577821 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:30.378289 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019e4e27-2226-4ef8-9388-8bd8335b7c9a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf9eed62c8f1409282c2af6488426a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:30.379066 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e4e27-2226-4ef8-9388-8bd8335b7c9a service swift] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:30.379469 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e4e27-2226-4ef8-9388-8bd8335b7c9a service swift] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:30.379893 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e4e27-2226-4ef8-9388-8bd8335b7c9a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:30.381764 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-94548676-43fd-4c16-8e3d-7be41aab4fae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5423db71126847609bb438d72b6351fa'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:30.389281 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019e4e27-2226-4ef8-9388-8bd8335b7c9a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:30.390598 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-94548676-43fd-4c16-8e3d-7be41aab4fae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5423db71126847609bb438d72b6351fa'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:30.390993 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019e4e27-2226-4ef8-9388-8bd8335b7c9a service swift] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:30.401070 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-94548676-43fd-4c16-8e3d-7be41aab4fae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5423db71126847609bb438d72b6351fa'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:30.401732 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1295/2607] 10.222.0.23 () {60 vars in 1384 bytes} [Fri May 6 15:07:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 06 15:07:30.408765 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-94548676-43fd-4c16-8e3d-7be41aab4fae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5423db71126847609bb438d72b6351fa'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:30.409849 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-94548676-43fd-4c16-8e3d-7be41aab4fae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5423db71126847609bb438d72b6351fa'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:30.430929 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1313/2608] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:07:30 2022] DELETE /identity/v3/users/5423db71126847609bb438d72b6351fa => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 15:07:30.435701 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241997ce-9517-485d-a890-05594e309912 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:30.436437 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241997ce-9517-485d-a890-05594e309912 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:30.436728 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241997ce-9517-485d-a890-05594e309912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:30.456355 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-241997ce-9517-485d-a890-05594e309912 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:07:30.457680 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-241997ce-9517-485d-a890-05594e309912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:30.537397 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1296/2609] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:07:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 15:07:30.549207 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94548676-43fd-4c16-8e3d-7be41aab4fae admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:30.551652 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-94548676-43fd-4c16-8e3d-7be41aab4fae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:30.624959 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94548676-43fd-4c16-8e3d-7be41aab4fae admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:30.628961 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35854ed-51c5-4143-ac61-b5724b67c245 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:30.629653 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35854ed-51c5-4143-ac61-b5724b67c245 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:30.629963 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35854ed-51c5-4143-ac61-b5724b67c245 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:30.630244 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35854ed-51c5-4143-ac61-b5724b67c245 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:30.641563 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35854ed-51c5-4143-ac61-b5724b67c245 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:30.643030 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35854ed-51c5-4143-ac61-b5724b67c245 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:30.655403 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1314/2610] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:07:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:07:31.321207 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241997ce-9517-485d-a890-05594e309912 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:31.323283 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-241997ce-9517-485d-a890-05594e309912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:31.336914 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241997ce-9517-485d-a890-05594e309912 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:31.341850 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8fc1a1-2a78-4021-a7a9-9634ce262623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:31.343029 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fc1a1-2a78-4021-a7a9-9634ce262623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:31.343314 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fc1a1-2a78-4021-a7a9-9634ce262623 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:31.343589 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fc1a1-2a78-4021-a7a9-9634ce262623 admin admin] PATH_INFO: `/v3/projects/5e3b290a4b66445d8fb970821c12ce0f` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:31.344889 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8fc1a1-2a78-4021-a7a9-9634ce262623 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e3b290a4b66445d8fb970821c12ce0f)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:31.346856 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8fc1a1-2a78-4021-a7a9-9634ce262623 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:31.395350 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8b8fc1a1-2a78-4021-a7a9-9634ce262623 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e3b290a4b66445d8fb970821c12ce0f', 'request_id': 'req-8b8fc1a1-2a78-4021-a7a9-9634ce262623'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:31.400616 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8b8fc1a1-2a78-4021-a7a9-9634ce262623 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e3b290a4b66445d8fb970821c12ce0f', 'request_id': 'req-8b8fc1a1-2a78-4021-a7a9-9634ce262623'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:31.409076 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8b8fc1a1-2a78-4021-a7a9-9634ce262623 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e3b290a4b66445d8fb970821c12ce0f', 'request_id': 'req-8b8fc1a1-2a78-4021-a7a9-9634ce262623'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:31.421001 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8b8fc1a1-2a78-4021-a7a9-9634ce262623 admin admin] The token cache is being invalidate because project 5e3b290a4b66445d8fb970821c12ce0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:07:31.421292 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8b8fc1a1-2a78-4021-a7a9-9634ce262623 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:31.421949 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-8b8fc1a1-2a78-4021-a7a9-9634ce262623 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:31.423499 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1297/2611] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:07:31 2022] DELETE /identity/v3/projects/5e3b290a4b66445d8fb970821c12ce0f => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:07:31.427703 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35854ed-51c5-4143-ac61-b5724b67c245 service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:31.439196 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35854ed-51c5-4143-ac61-b5724b67c245 service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:31.496969 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d25cbce-4adc-4dd1-9ef9-83979df5e440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:31.497907 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25cbce-4adc-4dd1-9ef9-83979df5e440 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:31.498706 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25cbce-4adc-4dd1-9ef9-83979df5e440 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:31.498927 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25cbce-4adc-4dd1-9ef9-83979df5e440 admin admin] PATH_INFO: `/v3/users/7ba2e24bd4484fe4a8edecc32d10d5dd` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:31.500175 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d25cbce-4adc-4dd1-9ef9-83979df5e440 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ba2e24bd4484fe4a8edecc32d10d5dd)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:31.502156 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d25cbce-4adc-4dd1-9ef9-83979df5e440 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:31.553205 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-7d25cbce-4adc-4dd1-9ef9-83979df5e440 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ba2e24bd4484fe4a8edecc32d10d5dd'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:31.560672 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-7d25cbce-4adc-4dd1-9ef9-83979df5e440 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ba2e24bd4484fe4a8edecc32d10d5dd'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:31.569139 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-7d25cbce-4adc-4dd1-9ef9-83979df5e440 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ba2e24bd4484fe4a8edecc32d10d5dd'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:31.569925 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-7d25cbce-4adc-4dd1-9ef9-83979df5e440 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ba2e24bd4484fe4a8edecc32d10d5dd'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:31.570544 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-7d25cbce-4adc-4dd1-9ef9-83979df5e440 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ba2e24bd4484fe4a8edecc32d10d5dd'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:31.588769 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1315/2612] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:07:31 2022] DELETE /identity/v3/users/7ba2e24bd4484fe4a8edecc32d10d5dd => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:07:31.940647 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8fc1a1-2a78-4021-a7a9-9634ce262623 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:31.942105 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-8b8fc1a1-2a78-4021-a7a9-9634ce262623 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:31.958914 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8fc1a1-2a78-4021-a7a9-9634ce262623 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:32.019827 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4097d0-df2b-40e5-843b-5383080c906f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:32.020733 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4097d0-df2b-40e5-843b-5383080c906f admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:32.021016 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4097d0-df2b-40e5-843b-5383080c906f admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:32.021291 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4097d0-df2b-40e5-843b-5383080c906f admin admin] PATH_INFO: `/v3/projects/05c2eec00c124967bf9f9fdbcbc03938` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:32.022474 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4097d0-df2b-40e5-843b-5383080c906f admin admin] RBAC: Authorizing `identity:delete_project(project_id=05c2eec00c124967bf9f9fdbcbc03938)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:32.024404 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4097d0-df2b-40e5-843b-5383080c906f admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:32.071863 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-bb4097d0-df2b-40e5-843b-5383080c906f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05c2eec00c124967bf9f9fdbcbc03938', 'request_id': 'req-bb4097d0-df2b-40e5-843b-5383080c906f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:32.078738 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-bb4097d0-df2b-40e5-843b-5383080c906f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05c2eec00c124967bf9f9fdbcbc03938', 'request_id': 'req-bb4097d0-df2b-40e5-843b-5383080c906f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:32.086243 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-bb4097d0-df2b-40e5-843b-5383080c906f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05c2eec00c124967bf9f9fdbcbc03938', 'request_id': 'req-bb4097d0-df2b-40e5-843b-5383080c906f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:32.098072 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-bb4097d0-df2b-40e5-843b-5383080c906f admin admin] The token cache is being invalidate because project 05c2eec00c124967bf9f9fdbcbc03938 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:07:32.098357 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-bb4097d0-df2b-40e5-843b-5383080c906f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:32.099000 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-bb4097d0-df2b-40e5-843b-5383080c906f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:32.100528 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1298/2613] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:07:31 2022] DELETE /identity/v3/projects/05c2eec00c124967bf9f9fdbcbc03938 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:07:32.105364 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b30ba-760e-4e42-bb38-67ffc374b6b2 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:32.105655 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b30ba-760e-4e42-bb38-67ffc374b6b2 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:32.105936 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b30ba-760e-4e42-bb38-67ffc374b6b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:32.125547 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-6b2b30ba-760e-4e42-bb38-67ffc374b6b2 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:07:32.126739 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-6b2b30ba-760e-4e42-bb38-67ffc374b6b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:32.195786 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1316/2614] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:07:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:07:32.203598 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c88233-240f-48e7-bd95-bb6ab99ba617 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:32.203917 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c88233-240f-48e7-bd95-bb6ab99ba617 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:32.204203 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c88233-240f-48e7-bd95-bb6ab99ba617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:32.223357 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-42c88233-240f-48e7-bd95-bb6ab99ba617 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:07:32.224694 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-42c88233-240f-48e7-bd95-bb6ab99ba617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:32.234231 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1299/2615] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:07:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 15:07:32.238242 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2b30ba-760e-4e42-bb38-67ffc374b6b2 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:32.248923 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2b30ba-760e-4e42-bb38-67ffc374b6b2 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:32.253800 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65f793-96f5-4b5d-826a-6c8ad5e0d23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:32.254515 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65f793-96f5-4b5d-826a-6c8ad5e0d23e admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:32.254801 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65f793-96f5-4b5d-826a-6c8ad5e0d23e admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:32.255082 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65f793-96f5-4b5d-826a-6c8ad5e0d23e admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:32.255814 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b65f793-96f5-4b5d-826a-6c8ad5e0d23e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 15:07:32.256147 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b65f793-96f5-4b5d-826a-6c8ad5e0d23e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:32.257767 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b65f793-96f5-4b5d-826a-6c8ad5e0d23e admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:32.270882 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1317/2616] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 15:07:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 06 15:07:32.274992 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c88233-240f-48e7-bd95-bb6ab99ba617 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:32.285135 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c88233-240f-48e7-bd95-bb6ab99ba617 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:32.290328 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a0a61-e77d-47ca-89b6-b93b33fb46ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:32.291012 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a0a61-e77d-47ca-89b6-b93b33fb46ac admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:32.291295 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a0a61-e77d-47ca-89b6-b93b33fb46ac admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:32.291565 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a0a61-e77d-47ca-89b6-b93b33fb46ac admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:32.292276 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25a0a61-e77d-47ca-89b6-b93b33fb46ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:32.294265 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25a0a61-e77d-47ca-89b6-b93b33fb46ac admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:32.322020 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1300/2617] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:07:32 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:07:32.326816 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65f793-96f5-4b5d-826a-6c8ad5e0d23e admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:32.337583 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65f793-96f5-4b5d-826a-6c8ad5e0d23e admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:32.395984 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44620f8-a703-428b-8ec2-b7d3f6d3f228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:32.396705 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44620f8-a703-428b-8ec2-b7d3f6d3f228 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:32.396989 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44620f8-a703-428b-8ec2-b7d3f6d3f228 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:32.397260 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44620f8-a703-428b-8ec2-b7d3f6d3f228 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:32.397941 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44620f8-a703-428b-8ec2-b7d3f6d3f228 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:32.399869 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44620f8-a703-428b-8ec2-b7d3f6d3f228 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:32.420373 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1318/2618] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:07:32 2022] POST /identity/v3/users => generated 284 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 15:07:32.424586 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a0a61-e77d-47ca-89b6-b93b33fb46ac admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:32.434970 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a0a61-e77d-47ca-89b6-b93b33fb46ac admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:32.439673 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de194e58-f782-4788-8505-411b75157bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:32.440374 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de194e58-f782-4788-8505-411b75157bc9 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:32.440709 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de194e58-f782-4788-8505-411b75157bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:32.440991 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de194e58-f782-4788-8505-411b75157bc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:32.441715 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de194e58-f782-4788-8505-411b75157bc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:32.443312 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de194e58-f782-4788-8505-411b75157bc9 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:32.454779 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1301/2619] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:07:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:07:32.458787 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44620f8-a703-428b-8ec2-b7d3f6d3f228 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:32.470240 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44620f8-a703-428b-8ec2-b7d3f6d3f228 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:32.474962 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9903f32c-0337-4133-8780-ecb2419e3fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:32.475635 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903f32c-0337-4133-8780-ecb2419e3fb7 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:32.475917 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903f32c-0337-4133-8780-ecb2419e3fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:32.476193 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903f32c-0337-4133-8780-ecb2419e3fb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:32.476949 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9903f32c-0337-4133-8780-ecb2419e3fb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:32.478678 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9903f32c-0337-4133-8780-ecb2419e3fb7 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:32.490078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1319/2620] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:07:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:07:32.494320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de194e58-f782-4788-8505-411b75157bc9 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:32.505534 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de194e58-f782-4788-8505-411b75157bc9 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:32.510352 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f28772e-8512-4651-88e4-5b93a81b2488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:32.511399 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28772e-8512-4651-88e4-5b93a81b2488 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:32.511660 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28772e-8512-4651-88e4-5b93a81b2488 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:32.511853 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28772e-8512-4651-88e4-5b93a81b2488 admin admin] PATH_INFO: `/v3/projects/f45da4265391497789469fb73b82b596/users/b426722f1b124fde9db3dd6efb9e1371/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:32.527561 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f28772e-8512-4651-88e4-5b93a81b2488 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f45da4265391497789469fb73b82b596, user_id=b426722f1b124fde9db3dd6efb9e1371, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:32.530777 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f28772e-8512-4651-88e4-5b93a81b2488 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:32.547935 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1302/2621] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:07:32 2022] PUT /identity/v3/projects/f45da4265391497789469fb73b82b596/users/b426722f1b124fde9db3dd6efb9e1371/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:07:32.565760 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f6230-96ec-4dd1-80d7-6ac63440c87f None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:32.566075 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f6230-96ec-4dd1-80d7-6ac63440c87f None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:32.566337 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f6230-96ec-4dd1-80d7-6ac63440c87f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:32.596080 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-2c8f6230-96ec-4dd1-80d7-6ac63440c87f None None] MFA Rules not processed for user `b426722f1b124fde9db3dd6efb9e1371`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:07:32.597356 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-2c8f6230-96ec-4dd1-80d7-6ac63440c87f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:32.660404 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1320/2622] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:07:32 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:07:32.664954 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f28772e-8512-4651-88e4-5b93a81b2488 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:32.726401 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f28772e-8512-4651-88e4-5b93a81b2488 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:32.730932 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5338576-5bdf-4059-816b-b7cc5d5cceac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:32.731587 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5338576-5bdf-4059-816b-b7cc5d5cceac admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:32.731870 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5338576-5bdf-4059-816b-b7cc5d5cceac admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:32.732146 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5338576-5bdf-4059-816b-b7cc5d5cceac admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:32.733693 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5338576-5bdf-4059-816b-b7cc5d5cceac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:32.735610 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5338576-5bdf-4059-816b-b7cc5d5cceac admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:32.760310 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1303/2623] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:07:32 2022] POST /identity/v3/projects => generated 367 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:07:32.765231 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f6230-96ec-4dd1-80d7-6ac63440c87f None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:32.775976 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f6230-96ec-4dd1-80d7-6ac63440c87f None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:32.830797 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c508c-9629-433a-9194-3404ebf290af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:32.831486 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c508c-9629-433a-9194-3404ebf290af admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:32.831772 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c508c-9629-433a-9194-3404ebf290af admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:32.832049 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c508c-9629-433a-9194-3404ebf290af admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:32.832757 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49c508c-9629-433a-9194-3404ebf290af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:32.834658 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49c508c-9629-433a-9194-3404ebf290af admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:32.854038 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1321/2624] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:07:32 2022] POST /identity/v3/users => generated 290 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:07:32.858288 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5338576-5bdf-4059-816b-b7cc5d5cceac admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:32.868813 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5338576-5bdf-4059-816b-b7cc5d5cceac admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:32.874323 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c809b-37fa-4c52-aad8-7a6a256b24b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:32.875045 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c809b-37fa-4c52-aad8-7a6a256b24b0 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:32.875220 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c809b-37fa-4c52-aad8-7a6a256b24b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:32.875491 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c809b-37fa-4c52-aad8-7a6a256b24b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:32.876046 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6c809b-37fa-4c52-aad8-7a6a256b24b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:32.877719 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6c809b-37fa-4c52-aad8-7a6a256b24b0 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:32.888141 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1304/2625] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:07:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:07:32.893746 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c508c-9629-433a-9194-3404ebf290af admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:32.904068 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c508c-9629-433a-9194-3404ebf290af admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:32.909543 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3bfa7-7e9d-4b95-aba2-6fd4c9d5929f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:32.910433 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3bfa7-7e9d-4b95-aba2-6fd4c9d5929f admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:32.910770 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3bfa7-7e9d-4b95-aba2-6fd4c9d5929f admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:32.911044 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3bfa7-7e9d-4b95-aba2-6fd4c9d5929f admin admin] PATH_INFO: `/v3/projects/73512e36c60e4b0681bd3bee109b8e8c/users/8c6ca0d1dc3c4af6b6a88458def4630f/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:32.927154 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c3bfa7-7e9d-4b95-aba2-6fd4c9d5929f admin admin] RBAC: Authorizing `identity:create_grant(project_id=73512e36c60e4b0681bd3bee109b8e8c, user_id=8c6ca0d1dc3c4af6b6a88458def4630f, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:32.929826 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c3bfa7-7e9d-4b95-aba2-6fd4c9d5929f admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:32.944689 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1322/2626] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:07:32 2022] PUT /identity/v3/projects/73512e36c60e4b0681bd3bee109b8e8c/users/8c6ca0d1dc3c4af6b6a88458def4630f/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:07:32.948744 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c809b-37fa-4c52-aad8-7a6a256b24b0 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:32.958556 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c809b-37fa-4c52-aad8-7a6a256b24b0 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:33.014814 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cd77aa-2d93-44c6-96e7-7cc0df82084f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:33.015480 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd77aa-2d93-44c6-96e7-7cc0df82084f admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:33.015765 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd77aa-2d93-44c6-96e7-7cc0df82084f admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:33.016044 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd77aa-2d93-44c6-96e7-7cc0df82084f admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:33.016804 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cd77aa-2d93-44c6-96e7-7cc0df82084f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:33.018392 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cd77aa-2d93-44c6-96e7-7cc0df82084f admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:33.028248 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1305/2627] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:07:32 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:07:33.032778 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3bfa7-7e9d-4b95-aba2-6fd4c9d5929f admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:33.042812 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3bfa7-7e9d-4b95-aba2-6fd4c9d5929f admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:33.048229 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd1cc8-28a4-4a76-8106-d965a5cee835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:33.049141 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd1cc8-28a4-4a76-8106-d965a5cee835 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:33.049423 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd1cc8-28a4-4a76-8106-d965a5cee835 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:33.049810 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd1cc8-28a4-4a76-8106-d965a5cee835 admin admin] PATH_INFO: `/v3/projects/73512e36c60e4b0681bd3bee109b8e8c/users/8c6ca0d1dc3c4af6b6a88458def4630f/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:33.051747 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd1cc8-28a4-4a76-8106-d965a5cee835 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73512e36c60e4b0681bd3bee109b8e8c, user_id=8c6ca0d1dc3c4af6b6a88458def4630f, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:33.054396 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd1cc8-28a4-4a76-8106-d965a5cee835 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:33.065868 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1323/2628] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:07:33 2022] PUT /identity/v3/projects/73512e36c60e4b0681bd3bee109b8e8c/users/8c6ca0d1dc3c4af6b6a88458def4630f/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:07:33.076226 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26816ac4-9f65-4a2a-b910-9f51dd20d904 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:33.076524 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26816ac4-9f65-4a2a-b910-9f51dd20d904 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:33.076833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26816ac4-9f65-4a2a-b910-9f51dd20d904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:33.108842 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-26816ac4-9f65-4a2a-b910-9f51dd20d904 None None] MFA Rules not processed for user `8c6ca0d1dc3c4af6b6a88458def4630f`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:07:33.108842 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-26816ac4-9f65-4a2a-b910-9f51dd20d904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:33.174889 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1306/2629] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:07:33 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:07:33.186069 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd1cc8-28a4-4a76-8106-d965a5cee835 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:33.187785 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-06cd1cc8-28a4-4a76-8106-d965a5cee835 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:33.243679 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd1cc8-28a4-4a76-8106-d965a5cee835 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:33.247364 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b70cedf-835b-489a-831c-3be39d2df4b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:33.248675 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b70cedf-835b-489a-831c-3be39d2df4b6 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:33.249062 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b70cedf-835b-489a-831c-3be39d2df4b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:33.249500 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b70cedf-835b-489a-831c-3be39d2df4b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:33.258943 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b70cedf-835b-489a-831c-3be39d2df4b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:33.260372 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b70cedf-835b-489a-831c-3be39d2df4b6 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:33.270298 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1324/2630] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:07:33 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:07:34.643676 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26816ac4-9f65-4a2a-b910-9f51dd20d904 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:34.710289 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26816ac4-9f65-4a2a-b910-9f51dd20d904 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:34.714665 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f6abd6-492f-4a6e-92c3-6b7513fb2192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:34.715530 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6abd6-492f-4a6e-92c3-6b7513fb2192 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:34.715809 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6abd6-492f-4a6e-92c3-6b7513fb2192 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:34.716080 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6abd6-492f-4a6e-92c3-6b7513fb2192 admin admin] PATH_INFO: `/v3/users/b426722f1b124fde9db3dd6efb9e1371` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:34.718125 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f6abd6-492f-4a6e-92c3-6b7513fb2192 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b426722f1b124fde9db3dd6efb9e1371)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:34.720095 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f6abd6-492f-4a6e-92c3-6b7513fb2192 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:34.768747 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-17f6abd6-492f-4a6e-92c3-6b7513fb2192 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b426722f1b124fde9db3dd6efb9e1371'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:34.769397 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-17f6abd6-492f-4a6e-92c3-6b7513fb2192 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b426722f1b124fde9db3dd6efb9e1371'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:34.777199 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-17f6abd6-492f-4a6e-92c3-6b7513fb2192 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b426722f1b124fde9db3dd6efb9e1371'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:34.784791 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-17f6abd6-492f-4a6e-92c3-6b7513fb2192 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b426722f1b124fde9db3dd6efb9e1371'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:34.785460 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-17f6abd6-492f-4a6e-92c3-6b7513fb2192 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b426722f1b124fde9db3dd6efb9e1371'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:34.802429 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1307/2631] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:07:34 2022] DELETE /identity/v3/users/b426722f1b124fde9db3dd6efb9e1371 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:07:34.807045 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf4899-f43d-479e-bf17-6b5a4034a5c8 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:34.807526 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf4899-f43d-479e-bf17-6b5a4034a5c8 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:34.807526 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf4899-f43d-479e-bf17-6b5a4034a5c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:34.825491 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-05bf4899-f43d-479e-bf17-6b5a4034a5c8 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:07:34.826716 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-05bf4899-f43d-479e-bf17-6b5a4034a5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:34.885070 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1325/2632] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:07:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:07:34.895177 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f6abd6-492f-4a6e-92c3-6b7513fb2192 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:34.896593 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-17f6abd6-492f-4a6e-92c3-6b7513fb2192 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:34.952488 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f6abd6-492f-4a6e-92c3-6b7513fb2192 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:34.956167 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c92c10-5ab3-425c-93a3-29a39232452c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:34.956845 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c92c10-5ab3-425c-93a3-29a39232452c service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:34.957129 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c92c10-5ab3-425c-93a3-29a39232452c service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:34.957407 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c92c10-5ab3-425c-93a3-29a39232452c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:34.967284 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c92c10-5ab3-425c-93a3-29a39232452c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:34.968739 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c92c10-5ab3-425c-93a3-29a39232452c service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:34.978219 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1308/2633] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:07:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 15:07:35.253956 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf4899-f43d-479e-bf17-6b5a4034a5c8 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:35.255490 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-05bf4899-f43d-479e-bf17-6b5a4034a5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:35.270791 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf4899-f43d-479e-bf17-6b5a4034a5c8 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:35.275561 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1aa6cf-08f1-4390-baad-9641a9b27220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:35.276504 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1aa6cf-08f1-4390-baad-9641a9b27220 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:35.277838 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1aa6cf-08f1-4390-baad-9641a9b27220 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:35.278153 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1aa6cf-08f1-4390-baad-9641a9b27220 admin admin] PATH_INFO: `/v3/projects/f45da4265391497789469fb73b82b596` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:35.280946 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1aa6cf-08f1-4390-baad-9641a9b27220 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f45da4265391497789469fb73b82b596)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:35.282969 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1aa6cf-08f1-4390-baad-9641a9b27220 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:35.332946 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9a1aa6cf-08f1-4390-baad-9641a9b27220 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f45da4265391497789469fb73b82b596', 'request_id': 'req-9a1aa6cf-08f1-4390-baad-9641a9b27220'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:35.338557 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9a1aa6cf-08f1-4390-baad-9641a9b27220 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f45da4265391497789469fb73b82b596', 'request_id': 'req-9a1aa6cf-08f1-4390-baad-9641a9b27220'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:35.348875 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9a1aa6cf-08f1-4390-baad-9641a9b27220 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f45da4265391497789469fb73b82b596', 'request_id': 'req-9a1aa6cf-08f1-4390-baad-9641a9b27220'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:35.362228 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9a1aa6cf-08f1-4390-baad-9641a9b27220 admin admin] The token cache is being invalidate because project f45da4265391497789469fb73b82b596 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:07:35.362526 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9a1aa6cf-08f1-4390-baad-9641a9b27220 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:35.363218 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-9a1aa6cf-08f1-4390-baad-9641a9b27220 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:35.364821 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1326/2634] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:07:35 2022] DELETE /identity/v3/projects/f45da4265391497789469fb73b82b596 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:07:35.368860 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c92c10-5ab3-425c-93a3-29a39232452c service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:35.378962 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c92c10-5ab3-425c-93a3-29a39232452c service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:35.440096 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b579b4-c1ef-4c3d-b2d5-2103d27e6e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:35.441006 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b579b4-c1ef-4c3d-b2d5-2103d27e6e64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:35.441290 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b579b4-c1ef-4c3d-b2d5-2103d27e6e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:35.441563 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b579b4-c1ef-4c3d-b2d5-2103d27e6e64 admin admin] PATH_INFO: `/v3/users/8c6ca0d1dc3c4af6b6a88458def4630f` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:35.442792 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b579b4-c1ef-4c3d-b2d5-2103d27e6e64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c6ca0d1dc3c4af6b6a88458def4630f)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:35.444796 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b579b4-c1ef-4c3d-b2d5-2103d27e6e64 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:35.499100 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e5b579b4-c1ef-4c3d-b2d5-2103d27e6e64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c6ca0d1dc3c4af6b6a88458def4630f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:35.499100 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e5b579b4-c1ef-4c3d-b2d5-2103d27e6e64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c6ca0d1dc3c4af6b6a88458def4630f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:35.505910 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e5b579b4-c1ef-4c3d-b2d5-2103d27e6e64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c6ca0d1dc3c4af6b6a88458def4630f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:35.514729 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e5b579b4-c1ef-4c3d-b2d5-2103d27e6e64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c6ca0d1dc3c4af6b6a88458def4630f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:35.515494 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e5b579b4-c1ef-4c3d-b2d5-2103d27e6e64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c6ca0d1dc3c4af6b6a88458def4630f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:35.536795 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1309/2635] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:07:35 2022] DELETE /identity/v3/users/8c6ca0d1dc3c4af6b6a88458def4630f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:07:35.912267 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1aa6cf-08f1-4390-baad-9641a9b27220 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:35.912267 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-9a1aa6cf-08f1-4390-baad-9641a9b27220 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:35.924403 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1aa6cf-08f1-4390-baad-9641a9b27220 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:35.980727 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f338dae6-27b0-4312-be76-a6bfadd5433c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:35.981601 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338dae6-27b0-4312-be76-a6bfadd5433c admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:35.981885 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338dae6-27b0-4312-be76-a6bfadd5433c admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:35.982171 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338dae6-27b0-4312-be76-a6bfadd5433c admin admin] PATH_INFO: `/v3/projects/73512e36c60e4b0681bd3bee109b8e8c` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:35.983773 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f338dae6-27b0-4312-be76-a6bfadd5433c admin admin] RBAC: Authorizing `identity:delete_project(project_id=73512e36c60e4b0681bd3bee109b8e8c)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:35.985740 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f338dae6-27b0-4312-be76-a6bfadd5433c admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:36.032672 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f338dae6-27b0-4312-be76-a6bfadd5433c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73512e36c60e4b0681bd3bee109b8e8c', 'request_id': 'req-f338dae6-27b0-4312-be76-a6bfadd5433c'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:36.037964 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f338dae6-27b0-4312-be76-a6bfadd5433c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73512e36c60e4b0681bd3bee109b8e8c', 'request_id': 'req-f338dae6-27b0-4312-be76-a6bfadd5433c'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:36.045483 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f338dae6-27b0-4312-be76-a6bfadd5433c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73512e36c60e4b0681bd3bee109b8e8c', 'request_id': 'req-f338dae6-27b0-4312-be76-a6bfadd5433c'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:36.057159 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f338dae6-27b0-4312-be76-a6bfadd5433c admin admin] The token cache is being invalidate because project 73512e36c60e4b0681bd3bee109b8e8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:07:36.057451 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f338dae6-27b0-4312-be76-a6bfadd5433c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:36.058115 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f338dae6-27b0-4312-be76-a6bfadd5433c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:36.059671 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1327/2636] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:07:35 2022] DELETE /identity/v3/projects/73512e36c60e4b0681bd3bee109b8e8c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:07:36.065122 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6004c883-321b-4df3-b518-e47cdba1438e None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:36.065659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6004c883-321b-4df3-b518-e47cdba1438e None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:36.065659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6004c883-321b-4df3-b518-e47cdba1438e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:36.084677 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-6004c883-321b-4df3-b518-e47cdba1438e None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:07:36.085884 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-6004c883-321b-4df3-b518-e47cdba1438e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:36.142993 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1310/2637] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:07:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:07:36.150659 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b0449-b93f-4408-acaa-0af2a105189b None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:36.150952 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b0449-b93f-4408-acaa-0af2a105189b None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:36.151234 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b0449-b93f-4408-acaa-0af2a105189b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:36.172378 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-048b0449-b93f-4408-acaa-0af2a105189b None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:07:36.173612 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-048b0449-b93f-4408-acaa-0af2a105189b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:36.185642 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1328/2638] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:07:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:07:36.190016 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6004c883-321b-4df3-b518-e47cdba1438e None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:36.199731 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6004c883-321b-4df3-b518-e47cdba1438e None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:36.205114 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d95c3f-623d-4e1d-b8ff-0237f70a2c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:36.205778 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d95c3f-623d-4e1d-b8ff-0237f70a2c57 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:36.206061 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d95c3f-623d-4e1d-b8ff-0237f70a2c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:36.206362 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d95c3f-623d-4e1d-b8ff-0237f70a2c57 admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:36.207093 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d95c3f-623d-4e1d-b8ff-0237f70a2c57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 15:07:36.207406 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d95c3f-623d-4e1d-b8ff-0237f70a2c57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:36.209019 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d95c3f-623d-4e1d-b8ff-0237f70a2c57 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:36.221188 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1311/2639] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 15:07:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 15:07:36.225750 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048b0449-b93f-4408-acaa-0af2a105189b None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:36.234978 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048b0449-b93f-4408-acaa-0af2a105189b None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:36.239605 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b81398c-8e85-481e-8790-5a747be5872e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:36.240256 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81398c-8e85-481e-8790-5a747be5872e admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:36.240534 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81398c-8e85-481e-8790-5a747be5872e admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:36.240851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81398c-8e85-481e-8790-5a747be5872e admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:36.241525 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b81398c-8e85-481e-8790-5a747be5872e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:36.243399 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b81398c-8e85-481e-8790-5a747be5872e admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:36.270552 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1329/2640] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:07:36 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:07:36.274756 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d95c3f-623d-4e1d-b8ff-0237f70a2c57 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:36.285934 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d95c3f-623d-4e1d-b8ff-0237f70a2c57 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:36.340538 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b66498-afff-4d05-8714-0a03a697fe77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:36.341234 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b66498-afff-4d05-8714-0a03a697fe77 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:36.341513 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b66498-afff-4d05-8714-0a03a697fe77 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:36.341813 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b66498-afff-4d05-8714-0a03a697fe77 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:36.342509 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b66498-afff-4d05-8714-0a03a697fe77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:36.344407 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b66498-afff-4d05-8714-0a03a697fe77 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:36.364096 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1312/2641] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:07:36 2022] POST /identity/v3/users => generated 294 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:07:36.368375 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b81398c-8e85-481e-8790-5a747be5872e admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:36.379310 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b81398c-8e85-481e-8790-5a747be5872e admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:36.383912 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe4a36-de79-40cc-b4e6-22eedb1f16b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:36.384562 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe4a36-de79-40cc-b4e6-22eedb1f16b9 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:36.384898 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe4a36-de79-40cc-b4e6-22eedb1f16b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:36.385146 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe4a36-de79-40cc-b4e6-22eedb1f16b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:36.385862 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe4a36-de79-40cc-b4e6-22eedb1f16b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:36.387427 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe4a36-de79-40cc-b4e6-22eedb1f16b9 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:36.398845 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1330/2642] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:07:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 06 15:07:36.403206 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b66498-afff-4d05-8714-0a03a697fe77 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:36.413740 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b66498-afff-4d05-8714-0a03a697fe77 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:36.418451 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d9e04-a39d-477b-973c-cc86d44b4fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:36.419119 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d9e04-a39d-477b-973c-cc86d44b4fae admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:36.419396 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d9e04-a39d-477b-973c-cc86d44b4fae admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:36.419670 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d9e04-a39d-477b-973c-cc86d44b4fae admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:36.421139 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039d9e04-a39d-477b-973c-cc86d44b4fae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:36.422705 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039d9e04-a39d-477b-973c-cc86d44b4fae admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:36.433074 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1313/2643] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:07:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:07:36.437911 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe4a36-de79-40cc-b4e6-22eedb1f16b9 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:36.448766 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe4a36-de79-40cc-b4e6-22eedb1f16b9 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:36.454349 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c14abf-3469-4bc2-9c91-c14074b296f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:36.455239 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c14abf-3469-4bc2-9c91-c14074b296f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:36.455533 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c14abf-3469-4bc2-9c91-c14074b296f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:36.455813 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c14abf-3469-4bc2-9c91-c14074b296f2 admin admin] PATH_INFO: `/v3/projects/7c05487d54154ad0b5c8ba0de5bf999d/users/7d9594ef12cf42a6943a4719339519ff/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:36.470529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c14abf-3469-4bc2-9c91-c14074b296f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c05487d54154ad0b5c8ba0de5bf999d, user_id=7d9594ef12cf42a6943a4719339519ff, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:36.473210 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c14abf-3469-4bc2-9c91-c14074b296f2 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:36.486120 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1331/2644] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:07:36 2022] PUT /identity/v3/projects/7c05487d54154ad0b5c8ba0de5bf999d/users/7d9594ef12cf42a6943a4719339519ff/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:07:36.497418 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e61383-fbc6-4d87-af58-e7cc3522d30e None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:36.497732 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e61383-fbc6-4d87-af58-e7cc3522d30e None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:36.498577 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e61383-fbc6-4d87-af58-e7cc3522d30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:36.525321 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-07e61383-fbc6-4d87-af58-e7cc3522d30e None None] MFA Rules not processed for user `7d9594ef12cf42a6943a4719339519ff`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:07:36.526552 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-07e61383-fbc6-4d87-af58-e7cc3522d30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:36.592376 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1314/2645] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:07:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:07:36.596881 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c14abf-3469-4bc2-9c91-c14074b296f2 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:36.658107 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c14abf-3469-4bc2-9c91-c14074b296f2 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:36.662654 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf768b-ce8e-416c-908d-04baea2746b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:36.663311 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf768b-ce8e-416c-908d-04baea2746b8 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:36.663596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf768b-ce8e-416c-908d-04baea2746b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:36.663876 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf768b-ce8e-416c-908d-04baea2746b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:36.664562 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbf768b-ce8e-416c-908d-04baea2746b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:36.666496 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbf768b-ce8e-416c-908d-04baea2746b8 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:36.695914 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1332/2646] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:07:36 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:07:36.699917 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e61383-fbc6-4d87-af58-e7cc3522d30e None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:36.710616 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e61383-fbc6-4d87-af58-e7cc3522d30e None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:36.766580 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d69535-d839-41ed-a07e-b153d2d83642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:36.767265 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d69535-d839-41ed-a07e-b153d2d83642 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:36.767550 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d69535-d839-41ed-a07e-b153d2d83642 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:36.767829 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d69535-d839-41ed-a07e-b153d2d83642 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:36.768532 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d69535-d839-41ed-a07e-b153d2d83642 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:36.770502 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d69535-d839-41ed-a07e-b153d2d83642 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:36.790632 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1315/2647] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:07:36 2022] POST /identity/v3/users => generated 300 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 15:07:36.794615 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf768b-ce8e-416c-908d-04baea2746b8 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:36.805496 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf768b-ce8e-416c-908d-04baea2746b8 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:36.810143 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa66bab-2ed3-42b6-8536-1e7826aad14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:36.811559 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa66bab-2ed3-42b6-8536-1e7826aad14d admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:36.811846 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa66bab-2ed3-42b6-8536-1e7826aad14d admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:36.812123 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa66bab-2ed3-42b6-8536-1e7826aad14d admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:36.812879 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa66bab-2ed3-42b6-8536-1e7826aad14d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:36.814501 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa66bab-2ed3-42b6-8536-1e7826aad14d admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:36.824992 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1333/2648] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:07:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:07:36.829723 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d69535-d839-41ed-a07e-b153d2d83642 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:36.839672 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d69535-d839-41ed-a07e-b153d2d83642 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:36.845102 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea2298-d2b1-4c94-97d2-ddcc70fc8113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:36.845993 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea2298-d2b1-4c94-97d2-ddcc70fc8113 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:36.846272 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea2298-d2b1-4c94-97d2-ddcc70fc8113 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:36.846542 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea2298-d2b1-4c94-97d2-ddcc70fc8113 admin admin] PATH_INFO: `/v3/projects/7f053bb347ce4b89a0d39261f1f1eb90/users/bfaa8cd3d6a54932a060ba03ba523bf6/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:36.861504 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ea2298-d2b1-4c94-97d2-ddcc70fc8113 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f053bb347ce4b89a0d39261f1f1eb90, user_id=bfaa8cd3d6a54932a060ba03ba523bf6, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:36.864149 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ea2298-d2b1-4c94-97d2-ddcc70fc8113 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:36.877418 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1316/2649] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:07:36 2022] PUT /identity/v3/projects/7f053bb347ce4b89a0d39261f1f1eb90/users/bfaa8cd3d6a54932a060ba03ba523bf6/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:07:36.881836 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa66bab-2ed3-42b6-8536-1e7826aad14d admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:36.892333 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa66bab-2ed3-42b6-8536-1e7826aad14d admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:36.949631 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45b45c-e1a8-4be4-bc5f-b11967b5a3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:36.950365 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45b45c-e1a8-4be4-bc5f-b11967b5a3d0 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:36.950651 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45b45c-e1a8-4be4-bc5f-b11967b5a3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:36.951505 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45b45c-e1a8-4be4-bc5f-b11967b5a3d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:36.952299 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a45b45c-e1a8-4be4-bc5f-b11967b5a3d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:36.953927 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a45b45c-e1a8-4be4-bc5f-b11967b5a3d0 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:36.964143 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1334/2650] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:07:36 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:07:36.969117 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea2298-d2b1-4c94-97d2-ddcc70fc8113 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:36.979369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea2298-d2b1-4c94-97d2-ddcc70fc8113 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:36.984342 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788d3de-12fc-4ff1-883d-25a0bc3bc03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:36.985268 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788d3de-12fc-4ff1-883d-25a0bc3bc03c admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:36.985632 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788d3de-12fc-4ff1-883d-25a0bc3bc03c admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:36.985911 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788d3de-12fc-4ff1-883d-25a0bc3bc03c admin admin] PATH_INFO: `/v3/projects/7f053bb347ce4b89a0d39261f1f1eb90/users/bfaa8cd3d6a54932a060ba03ba523bf6/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:36.987908 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d788d3de-12fc-4ff1-883d-25a0bc3bc03c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f053bb347ce4b89a0d39261f1f1eb90, user_id=bfaa8cd3d6a54932a060ba03ba523bf6, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:36.990576 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d788d3de-12fc-4ff1-883d-25a0bc3bc03c admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:37.001608 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1317/2651] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:07:36 2022] PUT /identity/v3/projects/7f053bb347ce4b89a0d39261f1f1eb90/users/bfaa8cd3d6a54932a060ba03ba523bf6/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:07:37.012058 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8958b13c-291f-4785-afc2-93b5b65590a8 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:37.012388 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8958b13c-291f-4785-afc2-93b5b65590a8 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:37.012672 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8958b13c-291f-4785-afc2-93b5b65590a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:37.042612 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-8958b13c-291f-4785-afc2-93b5b65590a8 None None] MFA Rules not processed for user `bfaa8cd3d6a54932a060ba03ba523bf6`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:07:37.043835 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-8958b13c-291f-4785-afc2-93b5b65590a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:37.115472 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1335/2652] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:07:37 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:07:37.127168 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788d3de-12fc-4ff1-883d-25a0bc3bc03c admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:37.129084 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d788d3de-12fc-4ff1-883d-25a0bc3bc03c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:37.190463 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788d3de-12fc-4ff1-883d-25a0bc3bc03c admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:37.194255 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd3d9c-18dc-4d7c-8890-b187ef399dc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:37.194891 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd3d9c-18dc-4d7c-8890-b187ef399dc9 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:37.195189 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd3d9c-18dc-4d7c-8890-b187ef399dc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:37.195459 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd3d9c-18dc-4d7c-8890-b187ef399dc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:37.205022 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bd3d9c-18dc-4d7c-8890-b187ef399dc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:37.206443 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bd3d9c-18dc-4d7c-8890-b187ef399dc9 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:37.215620 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1318/2653] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:07:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:07:37.280906 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8958b13c-291f-4785-afc2-93b5b65590a8 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:37.351089 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8958b13c-291f-4785-afc2-93b5b65590a8 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:37.355627 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc3f970-f76f-4a68-b330-9a0812013fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:37.356496 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc3f970-f76f-4a68-b330-9a0812013fe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:37.356812 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc3f970-f76f-4a68-b330-9a0812013fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:37.357089 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc3f970-f76f-4a68-b330-9a0812013fe9 admin admin] PATH_INFO: `/v3/users/7d9594ef12cf42a6943a4719339519ff` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:37.359004 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc3f970-f76f-4a68-b330-9a0812013fe9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d9594ef12cf42a6943a4719339519ff)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:37.360947 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc3f970-f76f-4a68-b330-9a0812013fe9 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:37.407537 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-8fc3f970-f76f-4a68-b330-9a0812013fe9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d9594ef12cf42a6943a4719339519ff'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:37.414859 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-8fc3f970-f76f-4a68-b330-9a0812013fe9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d9594ef12cf42a6943a4719339519ff'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:37.422409 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-8fc3f970-f76f-4a68-b330-9a0812013fe9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d9594ef12cf42a6943a4719339519ff'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:37.423089 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-8fc3f970-f76f-4a68-b330-9a0812013fe9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d9594ef12cf42a6943a4719339519ff'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:37.423701 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-8fc3f970-f76f-4a68-b330-9a0812013fe9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d9594ef12cf42a6943a4719339519ff'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:37.440261 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1336/2654] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:07:37 2022] DELETE /identity/v3/users/7d9594ef12cf42a6943a4719339519ff => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:07:37.444833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bedfef2-3622-48b2-be66-84d1a1b6f7d8 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:37.445070 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bedfef2-3622-48b2-be66-84d1a1b6f7d8 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:37.445347 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bedfef2-3622-48b2-be66-84d1a1b6f7d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:37.463644 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-7bedfef2-3622-48b2-be66-84d1a1b6f7d8 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:07:37.464917 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-7bedfef2-3622-48b2-be66-84d1a1b6f7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:37.522408 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1319/2655] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:07:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:07:37.533527 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc3f970-f76f-4a68-b330-9a0812013fe9 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:37.534949 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-8fc3f970-f76f-4a68-b330-9a0812013fe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:37.592437 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc3f970-f76f-4a68-b330-9a0812013fe9 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:37.596254 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1842205d-008b-45da-8560-a4d3ed699742 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:37.596939 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842205d-008b-45da-8560-a4d3ed699742 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:37.597221 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842205d-008b-45da-8560-a4d3ed699742 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:37.597492 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842205d-008b-45da-8560-a4d3ed699742 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:37.606812 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1842205d-008b-45da-8560-a4d3ed699742 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:37.608296 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1842205d-008b-45da-8560-a4d3ed699742 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:37.618253 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1337/2656] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:07:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:07:37.891767 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bedfef2-3622-48b2-be66-84d1a1b6f7d8 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:37.893351 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-7bedfef2-3622-48b2-be66-84d1a1b6f7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:37.909914 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bedfef2-3622-48b2-be66-84d1a1b6f7d8 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:37.915486 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ab9a1-5749-45e8-b2e2-b7e97ffb09ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:37.916378 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ab9a1-5749-45e8-b2e2-b7e97ffb09ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:37.916680 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ab9a1-5749-45e8-b2e2-b7e97ffb09ed admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:37.916976 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ab9a1-5749-45e8-b2e2-b7e97ffb09ed admin admin] PATH_INFO: `/v3/projects/7c05487d54154ad0b5c8ba0de5bf999d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:37.918350 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ab9a1-5749-45e8-b2e2-b7e97ffb09ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c05487d54154ad0b5c8ba0de5bf999d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:37.920261 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ab9a1-5749-45e8-b2e2-b7e97ffb09ed admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:37.966405 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-ff5ab9a1-5749-45e8-b2e2-b7e97ffb09ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c05487d54154ad0b5c8ba0de5bf999d', 'request_id': 'req-ff5ab9a1-5749-45e8-b2e2-b7e97ffb09ed'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:37.971982 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-ff5ab9a1-5749-45e8-b2e2-b7e97ffb09ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c05487d54154ad0b5c8ba0de5bf999d', 'request_id': 'req-ff5ab9a1-5749-45e8-b2e2-b7e97ffb09ed'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:37.979870 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-ff5ab9a1-5749-45e8-b2e2-b7e97ffb09ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c05487d54154ad0b5c8ba0de5bf999d', 'request_id': 'req-ff5ab9a1-5749-45e8-b2e2-b7e97ffb09ed'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:37.991384 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-ff5ab9a1-5749-45e8-b2e2-b7e97ffb09ed admin admin] The token cache is being invalidate because project 7c05487d54154ad0b5c8ba0de5bf999d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:07:37.991670 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-ff5ab9a1-5749-45e8-b2e2-b7e97ffb09ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:37.992337 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-ff5ab9a1-5749-45e8-b2e2-b7e97ffb09ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:37.993902 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1320/2657] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:07:37 2022] DELETE /identity/v3/projects/7c05487d54154ad0b5c8ba0de5bf999d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:07:37.998645 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1842205d-008b-45da-8560-a4d3ed699742 service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:38.008974 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1842205d-008b-45da-8560-a4d3ed699742 service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:38.066766 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642b490-a12b-4433-b3ed-d3e229c0ce99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:38.067643 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642b490-a12b-4433-b3ed-d3e229c0ce99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:38.067926 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642b490-a12b-4433-b3ed-d3e229c0ce99 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:38.068209 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642b490-a12b-4433-b3ed-d3e229c0ce99 admin admin] PATH_INFO: `/v3/users/bfaa8cd3d6a54932a060ba03ba523bf6` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:38.070244 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b642b490-a12b-4433-b3ed-d3e229c0ce99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfaa8cd3d6a54932a060ba03ba523bf6)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:38.072150 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b642b490-a12b-4433-b3ed-d3e229c0ce99 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:38.120763 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b642b490-a12b-4433-b3ed-d3e229c0ce99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfaa8cd3d6a54932a060ba03ba523bf6'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:38.130984 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b642b490-a12b-4433-b3ed-d3e229c0ce99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfaa8cd3d6a54932a060ba03ba523bf6'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:38.142591 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b642b490-a12b-4433-b3ed-d3e229c0ce99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfaa8cd3d6a54932a060ba03ba523bf6'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:38.143670 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b642b490-a12b-4433-b3ed-d3e229c0ce99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfaa8cd3d6a54932a060ba03ba523bf6'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:38.144620 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b642b490-a12b-4433-b3ed-d3e229c0ce99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfaa8cd3d6a54932a060ba03ba523bf6'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:38.175051 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1338/2658] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:07:37 2022] DELETE /identity/v3/users/bfaa8cd3d6a54932a060ba03ba523bf6 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:07:38.461787 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ab9a1-5749-45e8-b2e2-b7e97ffb09ed admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:38.463907 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ff5ab9a1-5749-45e8-b2e2-b7e97ffb09ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:38.477636 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ab9a1-5749-45e8-b2e2-b7e97ffb09ed admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:38.539496 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd6655-1217-449e-a130-5e6b66d5baf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:38.540353 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd6655-1217-449e-a130-5e6b66d5baf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:38.540666 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd6655-1217-449e-a130-5e6b66d5baf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:38.540950 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd6655-1217-449e-a130-5e6b66d5baf1 admin admin] PATH_INFO: `/v3/projects/7f053bb347ce4b89a0d39261f1f1eb90` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:38.542122 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bd6655-1217-449e-a130-5e6b66d5baf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f053bb347ce4b89a0d39261f1f1eb90)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:38.544041 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bd6655-1217-449e-a130-5e6b66d5baf1 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:38.592197 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e8bd6655-1217-449e-a130-5e6b66d5baf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f053bb347ce4b89a0d39261f1f1eb90', 'request_id': 'req-e8bd6655-1217-449e-a130-5e6b66d5baf1'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:38.598003 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e8bd6655-1217-449e-a130-5e6b66d5baf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f053bb347ce4b89a0d39261f1f1eb90', 'request_id': 'req-e8bd6655-1217-449e-a130-5e6b66d5baf1'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:38.605677 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e8bd6655-1217-449e-a130-5e6b66d5baf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f053bb347ce4b89a0d39261f1f1eb90', 'request_id': 'req-e8bd6655-1217-449e-a130-5e6b66d5baf1'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:38.618060 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e8bd6655-1217-449e-a130-5e6b66d5baf1 admin admin] The token cache is being invalidate because project 7f053bb347ce4b89a0d39261f1f1eb90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:07:38.618347 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e8bd6655-1217-449e-a130-5e6b66d5baf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:38.618986 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-e8bd6655-1217-449e-a130-5e6b66d5baf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:38.620560 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1321/2659] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:07:38 2022] DELETE /identity/v3/projects/7f053bb347ce4b89a0d39261f1f1eb90 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:07:38.625587 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4940fb61-3088-4b97-a2b2-536081b3c9d7 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:38.625879 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4940fb61-3088-4b97-a2b2-536081b3c9d7 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:38.626161 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4940fb61-3088-4b97-a2b2-536081b3c9d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:38.643926 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-4940fb61-3088-4b97-a2b2-536081b3c9d7 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:07:38.645155 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-4940fb61-3088-4b97-a2b2-536081b3c9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:38.709293 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1339/2660] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:07:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:07:38.717486 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa96a3a-84ac-4599-8fee-8ee10a20a344 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:38.717783 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa96a3a-84ac-4599-8fee-8ee10a20a344 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:38.718065 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa96a3a-84ac-4599-8fee-8ee10a20a344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:38.736798 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-9aa96a3a-84ac-4599-8fee-8ee10a20a344 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:07:38.739516 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-9aa96a3a-84ac-4599-8fee-8ee10a20a344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:38.754359 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1322/2661] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:07:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 15:07:38.758590 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4940fb61-3088-4b97-a2b2-536081b3c9d7 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:38.768647 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4940fb61-3088-4b97-a2b2-536081b3c9d7 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:38.773311 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83029168-1ee1-44ea-bc4b-9d07cea0f848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:38.773972 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83029168-1ee1-44ea-bc4b-9d07cea0f848 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:38.774258 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83029168-1ee1-44ea-bc4b-9d07cea0f848 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:38.774536 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83029168-1ee1-44ea-bc4b-9d07cea0f848 admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:38.775255 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83029168-1ee1-44ea-bc4b-9d07cea0f848 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 15:07:38.775591 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83029168-1ee1-44ea-bc4b-9d07cea0f848 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:38.777215 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83029168-1ee1-44ea-bc4b-9d07cea0f848 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:38.790492 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1340/2662] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 15:07:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 15:07:38.794114 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa96a3a-84ac-4599-8fee-8ee10a20a344 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:38.804211 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa96a3a-84ac-4599-8fee-8ee10a20a344 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:38.808900 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd874026-8b75-47b8-a50b-1f98022c80f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:38.809556 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd874026-8b75-47b8-a50b-1f98022c80f6 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:38.809839 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd874026-8b75-47b8-a50b-1f98022c80f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:38.810113 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd874026-8b75-47b8-a50b-1f98022c80f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:38.810833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd874026-8b75-47b8-a50b-1f98022c80f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:38.812867 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd874026-8b75-47b8-a50b-1f98022c80f6 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:38.838397 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1323/2663] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:07:38 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 15:07:38.843914 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83029168-1ee1-44ea-bc4b-9d07cea0f848 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:38.853951 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83029168-1ee1-44ea-bc4b-9d07cea0f848 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:38.914325 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e828d-814e-4fbb-96fd-ce0e4e54055b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:38.915002 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e828d-814e-4fbb-96fd-ce0e4e54055b admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:38.915287 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e828d-814e-4fbb-96fd-ce0e4e54055b admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:38.915566 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e828d-814e-4fbb-96fd-ce0e4e54055b admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:38.916256 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0e828d-814e-4fbb-96fd-ce0e4e54055b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:38.918244 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0e828d-814e-4fbb-96fd-ce0e4e54055b admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:38.941434 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1341/2664] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:07:38 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:07:38.945687 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd874026-8b75-47b8-a50b-1f98022c80f6 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:38.956285 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd874026-8b75-47b8-a50b-1f98022c80f6 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:38.960852 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36f914-6021-4119-837d-4234a85e7d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:38.961507 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36f914-6021-4119-837d-4234a85e7d27 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:38.961787 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36f914-6021-4119-837d-4234a85e7d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:38.962064 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36f914-6021-4119-837d-4234a85e7d27 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:38.962780 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c36f914-6021-4119-837d-4234a85e7d27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:38.964383 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c36f914-6021-4119-837d-4234a85e7d27 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:38.975356 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1324/2665] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:07:38.979608 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e828d-814e-4fbb-96fd-ce0e4e54055b admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:38.991439 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e828d-814e-4fbb-96fd-ce0e4e54055b admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:38.996993 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d386be-b02a-4800-b9b5-4dcefc085fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:38.997664 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d386be-b02a-4800-b9b5-4dcefc085fe7 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:38.997959 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d386be-b02a-4800-b9b5-4dcefc085fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:38.998247 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d386be-b02a-4800-b9b5-4dcefc085fe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:38.998971 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d386be-b02a-4800-b9b5-4dcefc085fe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:39.000585 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d386be-b02a-4800-b9b5-4dcefc085fe7 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:39.012170 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1342/2666] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:07:39.017202 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36f914-6021-4119-837d-4234a85e7d27 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:39.027060 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36f914-6021-4119-837d-4234a85e7d27 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:39.033170 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e846cd90-5100-494e-80d4-579514001163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:39.034041 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e846cd90-5100-494e-80d4-579514001163 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:39.034323 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e846cd90-5100-494e-80d4-579514001163 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:39.034598 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e846cd90-5100-494e-80d4-579514001163 admin admin] PATH_INFO: `/v3/projects/e1fbe15b7877468c8e8e03bc684fe07d/users/782c1a1f08f3475085894b47f83d1034/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:39.050486 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e846cd90-5100-494e-80d4-579514001163 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1fbe15b7877468c8e8e03bc684fe07d, user_id=782c1a1f08f3475085894b47f83d1034, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:39.053265 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e846cd90-5100-494e-80d4-579514001163 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:39.067026 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1325/2667] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:07:39 2022] PUT /identity/v3/projects/e1fbe15b7877468c8e8e03bc684fe07d/users/782c1a1f08f3475085894b47f83d1034/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 15:07:39.078977 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6a5fa-943d-4cd8-9af4-45f6fc4ba446 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:39.079272 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6a5fa-943d-4cd8-9af4-45f6fc4ba446 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:39.079551 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6a5fa-943d-4cd8-9af4-45f6fc4ba446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:39.106321 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-28d6a5fa-943d-4cd8-9af4-45f6fc4ba446 None None] MFA Rules not processed for user `782c1a1f08f3475085894b47f83d1034`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:07:39.107759 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-28d6a5fa-943d-4cd8-9af4-45f6fc4ba446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:39.172970 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1343/2668] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:07:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 15:07:39.177498 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e846cd90-5100-494e-80d4-579514001163 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:39.238135 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e846cd90-5100-494e-80d4-579514001163 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:39.242938 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d82d5-ad24-4ace-9f4e-0d9f1e5a77cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:39.243538 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d82d5-ad24-4ace-9f4e-0d9f1e5a77cf admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:39.243820 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d82d5-ad24-4ace-9f4e-0d9f1e5a77cf admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:39.244095 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d82d5-ad24-4ace-9f4e-0d9f1e5a77cf admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:39.244816 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97d82d5-ad24-4ace-9f4e-0d9f1e5a77cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:39.246764 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97d82d5-ad24-4ace-9f4e-0d9f1e5a77cf admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:39.272245 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1326/2669] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:07:39 2022] POST /identity/v3/projects => generated 381 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 15:07:39.276499 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d6a5fa-943d-4cd8-9af4-45f6fc4ba446 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:39.288537 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d6a5fa-943d-4cd8-9af4-45f6fc4ba446 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:39.354407 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485836a1-ef0a-42df-b882-b7dd3e2fade7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:39.355085 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485836a1-ef0a-42df-b882-b7dd3e2fade7 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:39.355370 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485836a1-ef0a-42df-b882-b7dd3e2fade7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:39.355646 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485836a1-ef0a-42df-b882-b7dd3e2fade7 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:39.356350 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485836a1-ef0a-42df-b882-b7dd3e2fade7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:39.359354 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485836a1-ef0a-42df-b882-b7dd3e2fade7 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:39.379364 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1344/2670] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:07:39 2022] POST /identity/v3/users => generated 297 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:07:39.383419 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d82d5-ad24-4ace-9f4e-0d9f1e5a77cf admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:39.395105 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d82d5-ad24-4ace-9f4e-0d9f1e5a77cf admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:39.400315 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f67590-86f3-4ada-a07a-34a6b16dd927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:39.400509 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f67590-86f3-4ada-a07a-34a6b16dd927 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:39.400813 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f67590-86f3-4ada-a07a-34a6b16dd927 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:39.401084 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f67590-86f3-4ada-a07a-34a6b16dd927 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:39.401832 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f67590-86f3-4ada-a07a-34a6b16dd927 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:39.403693 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f67590-86f3-4ada-a07a-34a6b16dd927 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:39.425699 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1327/2671] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 06 15:07:39.427361 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485836a1-ef0a-42df-b882-b7dd3e2fade7 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:39.439374 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485836a1-ef0a-42df-b882-b7dd3e2fade7 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:39.444408 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18ae52e-12af-4fe2-a11f-39a03f007152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:39.445332 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ae52e-12af-4fe2-a11f-39a03f007152 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:39.445611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ae52e-12af-4fe2-a11f-39a03f007152 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:39.445880 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ae52e-12af-4fe2-a11f-39a03f007152 admin admin] PATH_INFO: `/v3/projects/6925df778fea44f2ac7059f8a9f19422/users/2da07a15fd0142f6808866827256383b/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:39.464465 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18ae52e-12af-4fe2-a11f-39a03f007152 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6925df778fea44f2ac7059f8a9f19422, user_id=2da07a15fd0142f6808866827256383b, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:39.467932 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18ae52e-12af-4fe2-a11f-39a03f007152 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:39.481495 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1345/2672] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:07:39 2022] PUT /identity/v3/projects/6925df778fea44f2ac7059f8a9f19422/users/2da07a15fd0142f6808866827256383b/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:07:39.487365 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f67590-86f3-4ada-a07a-34a6b16dd927 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:39.504114 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f67590-86f3-4ada-a07a-34a6b16dd927 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:39.569267 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774fc40-3f9a-4d8d-9e42-cd8df5a42a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:39.569944 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774fc40-3f9a-4d8d-9e42-cd8df5a42a88 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:39.570230 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774fc40-3f9a-4d8d-9e42-cd8df5a42a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:39.570508 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774fc40-3f9a-4d8d-9e42-cd8df5a42a88 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:39.571226 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0774fc40-3f9a-4d8d-9e42-cd8df5a42a88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:39.572865 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0774fc40-3f9a-4d8d-9e42-cd8df5a42a88 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:39.582859 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1328/2673] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:07:39.587498 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18ae52e-12af-4fe2-a11f-39a03f007152 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:39.598537 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18ae52e-12af-4fe2-a11f-39a03f007152 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:39.603273 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4516ea92-d052-4ff2-a6c5-596c0565a6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:39.604166 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516ea92-d052-4ff2-a6c5-596c0565a6ac admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:39.604445 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516ea92-d052-4ff2-a6c5-596c0565a6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:39.604747 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516ea92-d052-4ff2-a6c5-596c0565a6ac admin admin] PATH_INFO: `/v3/projects/6925df778fea44f2ac7059f8a9f19422/users/2da07a15fd0142f6808866827256383b/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:39.606707 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4516ea92-d052-4ff2-a6c5-596c0565a6ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=6925df778fea44f2ac7059f8a9f19422, user_id=2da07a15fd0142f6808866827256383b, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:39.610405 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4516ea92-d052-4ff2-a6c5-596c0565a6ac admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:39.620777 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1346/2674] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:07:39 2022] PUT /identity/v3/projects/6925df778fea44f2ac7059f8a9f19422/users/2da07a15fd0142f6808866827256383b/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:07:39.631234 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b240e-d7fe-4894-8a5f-6062c658e736 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:39.631516 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b240e-d7fe-4894-8a5f-6062c658e736 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:39.631788 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b240e-d7fe-4894-8a5f-6062c658e736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:39.659153 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-d43b240e-d7fe-4894-8a5f-6062c658e736 None None] MFA Rules not processed for user `2da07a15fd0142f6808866827256383b`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:07:39.660361 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d43b240e-d7fe-4894-8a5f-6062c658e736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:39.729788 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1329/2675] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:07:39 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 15:07:39.742149 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4516ea92-d052-4ff2-a6c5-596c0565a6ac admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:39.743913 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-4516ea92-d052-4ff2-a6c5-596c0565a6ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:39.802550 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4516ea92-d052-4ff2-a6c5-596c0565a6ac admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:39.806262 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d8fa7-3b41-410b-8a5a-675c764e6c0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:39.806911 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d8fa7-3b41-410b-8a5a-675c764e6c0c service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:39.807200 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d8fa7-3b41-410b-8a5a-675c764e6c0c service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:39.807479 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d8fa7-3b41-410b-8a5a-675c764e6c0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:39.813468 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02349e3c-503e-4723-b267-cee9ea1797bf None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:39.813831 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02349e3c-503e-4723-b267-cee9ea1797bf None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:39.814176 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02349e3c-503e-4723-b267-cee9ea1797bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:39.816962 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d8fa7-3b41-410b-8a5a-675c764e6c0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:39.818402 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d8fa7-3b41-410b-8a5a-675c764e6c0c service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:39.829003 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1347/2676] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:07:39 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:07:39.834132 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-02349e3c-503e-4723-b267-cee9ea1797bf None None] MFA Rules not processed for user `4b3af1d82c33446e998f07bce54ef941`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:07:39.835320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-02349e3c-503e-4723-b267-cee9ea1797bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:39.891535 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1330/2677] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:07:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 15:07:40.139223 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d8fa7-3b41-410b-8a5a-675c764e6c0c service cinder] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:40.148171 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d8fa7-3b41-410b-8a5a-675c764e6c0c service cinder] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:40.157461 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad167922-91e0-4343-9731-24ac0e4ab009 tempest-VolumeQuotaClassesTest-1809586767 tempest-VolumeQuotaClassesTest-1809586767-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2da07a15fd0142f6808866827256383b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6925df778fea44f2ac7059f8a9f19422', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:40.158085 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad167922-91e0-4343-9731-24ac0e4ab009 tempest-VolumeQuotaClassesTest-1809586767 tempest-VolumeQuotaClassesTest-1809586767-project-admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:40.158342 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad167922-91e0-4343-9731-24ac0e4ab009 tempest-VolumeQuotaClassesTest-1809586767 tempest-VolumeQuotaClassesTest-1809586767-project-admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:40.159002 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad167922-91e0-4343-9731-24ac0e4ab009 tempest-VolumeQuotaClassesTest-1809586767 tempest-VolumeQuotaClassesTest-1809586767-project-admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:40.159303 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad167922-91e0-4343-9731-24ac0e4ab009 tempest-VolumeQuotaClassesTest-1809586767 tempest-VolumeQuotaClassesTest-1809586767-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 15:07:40.159633 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad167922-91e0-4343-9731-24ac0e4ab009 tempest-VolumeQuotaClassesTest-1809586767 tempest-VolumeQuotaClassesTest-1809586767-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:40.161238 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad167922-91e0-4343-9731-24ac0e4ab009 tempest-VolumeQuotaClassesTest-1809586767 tempest-VolumeQuotaClassesTest-1809586767-project-admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:40.175441 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1348/2678] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 15:07:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 15:07:40.180089 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02349e3c-503e-4723-b267-cee9ea1797bf None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:40.196108 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02349e3c-503e-4723-b267-cee9ea1797bf None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:40.200684 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c702be-fa8a-435a-a5bb-7c2c877074b7 tempest-VolumeQuotaClassesTest-1809586767 tempest-VolumeQuotaClassesTest-1809586767-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2da07a15fd0142f6808866827256383b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6925df778fea44f2ac7059f8a9f19422', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:40.201547 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c702be-fa8a-435a-a5bb-7c2c877074b7 tempest-VolumeQuotaClassesTest-1809586767 tempest-VolumeQuotaClassesTest-1809586767-project-admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:40.202036 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c702be-fa8a-435a-a5bb-7c2c877074b7 tempest-VolumeQuotaClassesTest-1809586767 tempest-VolumeQuotaClassesTest-1809586767-project-admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:40.202491 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c702be-fa8a-435a-a5bb-7c2c877074b7 tempest-VolumeQuotaClassesTest-1809586767 tempest-VolumeQuotaClassesTest-1809586767-project-admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:40.203184 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c702be-fa8a-435a-a5bb-7c2c877074b7 tempest-VolumeQuotaClassesTest-1809586767 tempest-VolumeQuotaClassesTest-1809586767-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:40.205119 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c702be-fa8a-435a-a5bb-7c2c877074b7 tempest-VolumeQuotaClassesTest-1809586767 tempest-VolumeQuotaClassesTest-1809586767-project-admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:40.234298 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1331/2679] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:07:40 2022] POST /identity/v3/projects => generated 354 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:07:40.238264 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad167922-91e0-4343-9731-24ac0e4ab009 tempest-VolumeQuotaClassesTest-1809586767 tempest-VolumeQuotaClassesTest-1809586767-project-admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:40.248204 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad167922-91e0-4343-9731-24ac0e4ab009 tempest-VolumeQuotaClassesTest-1809586767 tempest-VolumeQuotaClassesTest-1809586767-project-admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:40.306123 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d29766-10dd-4477-9c19-2b2b0bc5f56e tempest-VolumeQuotaClassesTest-1809586767 tempest-VolumeQuotaClassesTest-1809586767-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2da07a15fd0142f6808866827256383b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6925df778fea44f2ac7059f8a9f19422', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:40.306805 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d29766-10dd-4477-9c19-2b2b0bc5f56e tempest-VolumeQuotaClassesTest-1809586767 tempest-VolumeQuotaClassesTest-1809586767-project-admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:40.307096 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d29766-10dd-4477-9c19-2b2b0bc5f56e tempest-VolumeQuotaClassesTest-1809586767 tempest-VolumeQuotaClassesTest-1809586767-project-admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:40.307380 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d29766-10dd-4477-9c19-2b2b0bc5f56e tempest-VolumeQuotaClassesTest-1809586767 tempest-VolumeQuotaClassesTest-1809586767-project-admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:40.308107 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d29766-10dd-4477-9c19-2b2b0bc5f56e tempest-VolumeQuotaClassesTest-1809586767 tempest-VolumeQuotaClassesTest-1809586767-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 15:07:40.308424 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d29766-10dd-4477-9c19-2b2b0bc5f56e tempest-VolumeQuotaClassesTest-1809586767 tempest-VolumeQuotaClassesTest-1809586767-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:40.310049 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d29766-10dd-4477-9c19-2b2b0bc5f56e tempest-VolumeQuotaClassesTest-1809586767 tempest-VolumeQuotaClassesTest-1809586767-project-admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:40.322278 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1349/2680] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 15:07:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 15:07:40.358910 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c702be-fa8a-435a-a5bb-7c2c877074b7 tempest-VolumeQuotaClassesTest-1809586767 tempest-VolumeQuotaClassesTest-1809586767-project-admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:40.369130 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c702be-fa8a-435a-a5bb-7c2c877074b7 tempest-VolumeQuotaClassesTest-1809586767 tempest-VolumeQuotaClassesTest-1809586767-project-admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:40.373990 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff3d94-814f-43ce-9f7f-fd34f846cc3c tempest-VolumeQuotaClassesTest-1809586767 tempest-VolumeQuotaClassesTest-1809586767-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2da07a15fd0142f6808866827256383b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6925df778fea44f2ac7059f8a9f19422', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:40.374853 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff3d94-814f-43ce-9f7f-fd34f846cc3c tempest-VolumeQuotaClassesTest-1809586767 tempest-VolumeQuotaClassesTest-1809586767-project-admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:40.375140 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff3d94-814f-43ce-9f7f-fd34f846cc3c tempest-VolumeQuotaClassesTest-1809586767 tempest-VolumeQuotaClassesTest-1809586767-project-admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:40.375416 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff3d94-814f-43ce-9f7f-fd34f846cc3c tempest-VolumeQuotaClassesTest-1809586767 tempest-VolumeQuotaClassesTest-1809586767-project-admin] PATH_INFO: `/v3/projects/dce198c3516146999250daa4a3fec56b` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:40.377146 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff3d94-814f-43ce-9f7f-fd34f846cc3c tempest-VolumeQuotaClassesTest-1809586767 tempest-VolumeQuotaClassesTest-1809586767-project-admin] RBAC: Authorizing `identity:delete_project(project_id=dce198c3516146999250daa4a3fec56b)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:40.378548 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff3d94-814f-43ce-9f7f-fd34f846cc3c tempest-VolumeQuotaClassesTest-1809586767 tempest-VolumeQuotaClassesTest-1809586767-project-admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:40.427001 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-bfff3d94-814f-43ce-9f7f-fd34f846cc3c tempest-VolumeQuotaClassesTest-1809586767 tempest-VolumeQuotaClassesTest-1809586767-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dce198c3516146999250daa4a3fec56b', 'request_id': 'req-bfff3d94-814f-43ce-9f7f-fd34f846cc3c'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:40.432634 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-bfff3d94-814f-43ce-9f7f-fd34f846cc3c tempest-VolumeQuotaClassesTest-1809586767 tempest-VolumeQuotaClassesTest-1809586767-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dce198c3516146999250daa4a3fec56b', 'request_id': 'req-bfff3d94-814f-43ce-9f7f-fd34f846cc3c'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:40.441896 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-bfff3d94-814f-43ce-9f7f-fd34f846cc3c tempest-VolumeQuotaClassesTest-1809586767 tempest-VolumeQuotaClassesTest-1809586767-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dce198c3516146999250daa4a3fec56b', 'request_id': 'req-bfff3d94-814f-43ce-9f7f-fd34f846cc3c'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:40.454754 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-bfff3d94-814f-43ce-9f7f-fd34f846cc3c tempest-VolumeQuotaClassesTest-1809586767 tempest-VolumeQuotaClassesTest-1809586767-project-admin] The token cache is being invalidate because project dce198c3516146999250daa4a3fec56b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:07:40.455039 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-bfff3d94-814f-43ce-9f7f-fd34f846cc3c tempest-VolumeQuotaClassesTest-1809586767 tempest-VolumeQuotaClassesTest-1809586767-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:40.455711 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-bfff3d94-814f-43ce-9f7f-fd34f846cc3c tempest-VolumeQuotaClassesTest-1809586767 tempest-VolumeQuotaClassesTest-1809586767-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:40.457549 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1332/2681] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:07:40 2022] DELETE /identity/v3/projects/dce198c3516146999250daa4a3fec56b => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:07:40.701540 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d29766-10dd-4477-9c19-2b2b0bc5f56e tempest-VolumeQuotaClassesTest-1809586767 tempest-VolumeQuotaClassesTest-1809586767-project-admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:40.703562 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-10d29766-10dd-4477-9c19-2b2b0bc5f56e tempest-VolumeQuotaClassesTest-1809586767 tempest-VolumeQuotaClassesTest-1809586767-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:40.775361 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d29766-10dd-4477-9c19-2b2b0bc5f56e tempest-VolumeQuotaClassesTest-1809586767 tempest-VolumeQuotaClassesTest-1809586767-project-admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:40.832777 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa99a5ab-50bf-4b39-816a-2db49ac01ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:40.834289 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99a5ab-50bf-4b39-816a-2db49ac01ee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:40.834289 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99a5ab-50bf-4b39-816a-2db49ac01ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:40.834289 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99a5ab-50bf-4b39-816a-2db49ac01ee1 admin admin] PATH_INFO: `/v3/users/782c1a1f08f3475085894b47f83d1034` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:40.835447 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa99a5ab-50bf-4b39-816a-2db49ac01ee1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=782c1a1f08f3475085894b47f83d1034)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:40.837635 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa99a5ab-50bf-4b39-816a-2db49ac01ee1 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:40.886173 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-fa99a5ab-50bf-4b39-816a-2db49ac01ee1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '782c1a1f08f3475085894b47f83d1034'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:40.893798 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-fa99a5ab-50bf-4b39-816a-2db49ac01ee1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '782c1a1f08f3475085894b47f83d1034'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:40.900915 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-fa99a5ab-50bf-4b39-816a-2db49ac01ee1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '782c1a1f08f3475085894b47f83d1034'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:40.901605 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-fa99a5ab-50bf-4b39-816a-2db49ac01ee1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '782c1a1f08f3475085894b47f83d1034'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:40.902283 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-fa99a5ab-50bf-4b39-816a-2db49ac01ee1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '782c1a1f08f3475085894b47f83d1034'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:40.922561 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1350/2682] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:07:40 2022] DELETE /identity/v3/users/782c1a1f08f3475085894b47f83d1034 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:07:40.927130 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd5cf3-4e76-44b5-911d-a489ceaf448a None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:40.927675 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd5cf3-4e76-44b5-911d-a489ceaf448a None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:40.927675 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd5cf3-4e76-44b5-911d-a489ceaf448a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:40.945104 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-6ddd5cf3-4e76-44b5-911d-a489ceaf448a None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:07:40.946310 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-6ddd5cf3-4e76-44b5-911d-a489ceaf448a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:41.010635 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1333/2683] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:07:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:07:41.021701 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa99a5ab-50bf-4b39-816a-2db49ac01ee1 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:41.023118 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-fa99a5ab-50bf-4b39-816a-2db49ac01ee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:41.083658 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa99a5ab-50bf-4b39-816a-2db49ac01ee1 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:41.089038 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f29fa53-8562-45ee-9972-e412644cc1b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:41.089704 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f29fa53-8562-45ee-9972-e412644cc1b8 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:41.089987 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f29fa53-8562-45ee-9972-e412644cc1b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:41.090265 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f29fa53-8562-45ee-9972-e412644cc1b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:41.100629 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f29fa53-8562-45ee-9972-e412644cc1b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:41.102078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f29fa53-8562-45ee-9972-e412644cc1b8 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:41.113038 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1351/2684] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:07:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:07:41.404893 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd5cf3-4e76-44b5-911d-a489ceaf448a None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:41.406380 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-6ddd5cf3-4e76-44b5-911d-a489ceaf448a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:41.418276 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd5cf3-4e76-44b5-911d-a489ceaf448a None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:41.424543 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d62d3-95b8-4c75-8684-96839128f397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:41.425465 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d62d3-95b8-4c75-8684-96839128f397 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:41.425789 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d62d3-95b8-4c75-8684-96839128f397 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:41.426024 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d62d3-95b8-4c75-8684-96839128f397 admin admin] PATH_INFO: `/v3/projects/e1fbe15b7877468c8e8e03bc684fe07d` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:41.427229 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d62d3-95b8-4c75-8684-96839128f397 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1fbe15b7877468c8e8e03bc684fe07d)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:41.429185 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d62d3-95b8-4c75-8684-96839128f397 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:41.480383 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-503d62d3-95b8-4c75-8684-96839128f397 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1fbe15b7877468c8e8e03bc684fe07d', 'request_id': 'req-503d62d3-95b8-4c75-8684-96839128f397'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:41.487954 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-503d62d3-95b8-4c75-8684-96839128f397 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1fbe15b7877468c8e8e03bc684fe07d', 'request_id': 'req-503d62d3-95b8-4c75-8684-96839128f397'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:41.497906 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-503d62d3-95b8-4c75-8684-96839128f397 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1fbe15b7877468c8e8e03bc684fe07d', 'request_id': 'req-503d62d3-95b8-4c75-8684-96839128f397'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:41.509633 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-503d62d3-95b8-4c75-8684-96839128f397 admin admin] The token cache is being invalidate because project e1fbe15b7877468c8e8e03bc684fe07d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:07:41.509924 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-503d62d3-95b8-4c75-8684-96839128f397 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:41.511092 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-503d62d3-95b8-4c75-8684-96839128f397 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:41.512687 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1334/2685] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:07:41 2022] DELETE /identity/v3/projects/e1fbe15b7877468c8e8e03bc684fe07d => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:07:41.517409 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f29fa53-8562-45ee-9972-e412644cc1b8 service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:41.527841 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f29fa53-8562-45ee-9972-e412644cc1b8 service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:41.612643 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f535b6e5-03f4-4ad3-9ea2-465e1140591f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:41.613452 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535b6e5-03f4-4ad3-9ea2-465e1140591f admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:41.613736 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535b6e5-03f4-4ad3-9ea2-465e1140591f admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:41.614017 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535b6e5-03f4-4ad3-9ea2-465e1140591f admin admin] PATH_INFO: `/v3/users/2da07a15fd0142f6808866827256383b` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:41.615268 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f535b6e5-03f4-4ad3-9ea2-465e1140591f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2da07a15fd0142f6808866827256383b)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:41.617222 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f535b6e5-03f4-4ad3-9ea2-465e1140591f admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:41.671289 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f535b6e5-03f4-4ad3-9ea2-465e1140591f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2da07a15fd0142f6808866827256383b'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:41.679473 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f535b6e5-03f4-4ad3-9ea2-465e1140591f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2da07a15fd0142f6808866827256383b'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:41.689635 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f535b6e5-03f4-4ad3-9ea2-465e1140591f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2da07a15fd0142f6808866827256383b'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:41.690378 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f535b6e5-03f4-4ad3-9ea2-465e1140591f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2da07a15fd0142f6808866827256383b'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:41.690995 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f535b6e5-03f4-4ad3-9ea2-465e1140591f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2da07a15fd0142f6808866827256383b'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:41.712224 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1352/2686] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:07:41 2022] DELETE /identity/v3/users/2da07a15fd0142f6808866827256383b => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 15:07:42.436121 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d62d3-95b8-4c75-8684-96839128f397 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:42.440376 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-503d62d3-95b8-4c75-8684-96839128f397 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:42.467392 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d62d3-95b8-4c75-8684-96839128f397 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:42.536253 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc0a29-3636-4ac4-a9cf-bbd32d73454b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:42.537148 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc0a29-3636-4ac4-a9cf-bbd32d73454b admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:42.537430 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc0a29-3636-4ac4-a9cf-bbd32d73454b admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:42.537704 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc0a29-3636-4ac4-a9cf-bbd32d73454b admin admin] PATH_INFO: `/v3/projects/6925df778fea44f2ac7059f8a9f19422` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:42.538891 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cc0a29-3636-4ac4-a9cf-bbd32d73454b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6925df778fea44f2ac7059f8a9f19422)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:42.540842 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cc0a29-3636-4ac4-a9cf-bbd32d73454b admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:42.588316 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-02cc0a29-3636-4ac4-a9cf-bbd32d73454b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6925df778fea44f2ac7059f8a9f19422', 'request_id': 'req-02cc0a29-3636-4ac4-a9cf-bbd32d73454b'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:42.594497 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-02cc0a29-3636-4ac4-a9cf-bbd32d73454b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6925df778fea44f2ac7059f8a9f19422', 'request_id': 'req-02cc0a29-3636-4ac4-a9cf-bbd32d73454b'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:42.603883 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-02cc0a29-3636-4ac4-a9cf-bbd32d73454b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6925df778fea44f2ac7059f8a9f19422', 'request_id': 'req-02cc0a29-3636-4ac4-a9cf-bbd32d73454b'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:42.617509 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-02cc0a29-3636-4ac4-a9cf-bbd32d73454b admin admin] The token cache is being invalidate because project 6925df778fea44f2ac7059f8a9f19422 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:07:42.618150 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-02cc0a29-3636-4ac4-a9cf-bbd32d73454b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:42.619136 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-02cc0a29-3636-4ac4-a9cf-bbd32d73454b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:07:42.621806 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1335/2687] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:07:42 2022] DELETE /identity/v3/projects/6925df778fea44f2ac7059f8a9f19422 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:07:42.630138 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05b6ab-a6b0-402a-aa36-b99bcc72dfc7 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:42.630657 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05b6ab-a6b0-402a-aa36-b99bcc72dfc7 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:42.631149 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05b6ab-a6b0-402a-aa36-b99bcc72dfc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:42.651249 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-0e05b6ab-a6b0-402a-aa36-b99bcc72dfc7 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:07:42.652451 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-0e05b6ab-a6b0-402a-aa36-b99bcc72dfc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:42.719159 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1353/2688] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:07:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:07:42.727318 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0244e1-3199-49b4-8dab-a684615f186b None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:42.727526 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0244e1-3199-49b4-8dab-a684615f186b None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:42.727799 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0244e1-3199-49b4-8dab-a684615f186b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:42.746425 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-ee0244e1-3199-49b4-8dab-a684615f186b None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:07:42.747622 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ee0244e1-3199-49b4-8dab-a684615f186b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:42.756006 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1336/2689] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:07:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:07:42.760391 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05b6ab-a6b0-402a-aa36-b99bcc72dfc7 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:42.772877 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05b6ab-a6b0-402a-aa36-b99bcc72dfc7 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:42.778137 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8237a36-e124-4295-bf8f-0809c9b89a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:42.778924 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8237a36-e124-4295-bf8f-0809c9b89a5b admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:42.779781 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8237a36-e124-4295-bf8f-0809c9b89a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:42.780237 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8237a36-e124-4295-bf8f-0809c9b89a5b admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:42.781008 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8237a36-e124-4295-bf8f-0809c9b89a5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 15:07:42.781314 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8237a36-e124-4295-bf8f-0809c9b89a5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:42.782950 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8237a36-e124-4295-bf8f-0809c9b89a5b admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:42.799869 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1354/2690] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 15:07:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 15:07:42.801438 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0244e1-3199-49b4-8dab-a684615f186b None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:42.810931 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0244e1-3199-49b4-8dab-a684615f186b None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:42.816081 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea3521-0823-462c-b448-e4c474f5864d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:42.816782 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea3521-0823-462c-b448-e4c474f5864d admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:42.817061 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea3521-0823-462c-b448-e4c474f5864d admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:42.817356 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea3521-0823-462c-b448-e4c474f5864d admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:42.818053 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea3521-0823-462c-b448-e4c474f5864d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:42.820187 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea3521-0823-462c-b448-e4c474f5864d admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:42.854250 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1337/2691] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:07:42 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 15:07:42.858743 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8237a36-e124-4295-bf8f-0809c9b89a5b admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:42.870276 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8237a36-e124-4295-bf8f-0809c9b89a5b admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:42.939210 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071066bc-bcda-49b8-a709-130a77c49cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:42.939925 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071066bc-bcda-49b8-a709-130a77c49cfc admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:42.940231 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071066bc-bcda-49b8-a709-130a77c49cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:42.941393 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071066bc-bcda-49b8-a709-130a77c49cfc admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:42.941912 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071066bc-bcda-49b8-a709-130a77c49cfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:42.944082 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071066bc-bcda-49b8-a709-130a77c49cfc admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:42.966095 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1355/2692] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:07:42 2022] POST /identity/v3/users => generated 287 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:07:42.970856 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea3521-0823-462c-b448-e4c474f5864d admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:42.982258 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea3521-0823-462c-b448-e4c474f5864d admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:42.987495 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043c5d23-dfcd-4251-b4e8-07cc9e4aff28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:42.988261 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c5d23-dfcd-4251-b4e8-07cc9e4aff28 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:42.988668 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c5d23-dfcd-4251-b4e8-07cc9e4aff28 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:42.989024 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c5d23-dfcd-4251-b4e8-07cc9e4aff28 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:42.989836 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043c5d23-dfcd-4251-b4e8-07cc9e4aff28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:42.991525 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043c5d23-dfcd-4251-b4e8-07cc9e4aff28 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:43.003990 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1338/2693] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:07:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:07:43.008135 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071066bc-bcda-49b8-a709-130a77c49cfc admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:43.020290 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071066bc-bcda-49b8-a709-130a77c49cfc admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:43.025359 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96121679-cbff-4726-b687-2cab26730d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:43.026043 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96121679-cbff-4726-b687-2cab26730d9f admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:43.026340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96121679-cbff-4726-b687-2cab26730d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:43.026621 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96121679-cbff-4726-b687-2cab26730d9f admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:43.027355 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96121679-cbff-4726-b687-2cab26730d9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:43.029841 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96121679-cbff-4726-b687-2cab26730d9f admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:43.040406 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1356/2694] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:07:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:07:43.044623 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043c5d23-dfcd-4251-b4e8-07cc9e4aff28 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:43.056794 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043c5d23-dfcd-4251-b4e8-07cc9e4aff28 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:43.063093 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e5a42a-716c-4c77-8132-0af64c11007e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:43.063998 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5a42a-716c-4c77-8132-0af64c11007e admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:43.064278 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5a42a-716c-4c77-8132-0af64c11007e admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:43.064619 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5a42a-716c-4c77-8132-0af64c11007e admin admin] PATH_INFO: `/v3/projects/ad326e01031b493cb0c63533d25a2ec3/users/ad218437c31f443c8e009fffe6049edd/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:43.080338 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e5a42a-716c-4c77-8132-0af64c11007e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad326e01031b493cb0c63533d25a2ec3, user_id=ad218437c31f443c8e009fffe6049edd, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:43.083781 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e5a42a-716c-4c77-8132-0af64c11007e admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:43.102088 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1339/2695] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:07:43 2022] PUT /identity/v3/projects/ad326e01031b493cb0c63533d25a2ec3/users/ad218437c31f443c8e009fffe6049edd/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:07:43.108106 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac66bde-702e-48cd-bc43-83fbe83aead1 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:43.108483 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac66bde-702e-48cd-bc43-83fbe83aead1 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:43.108903 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac66bde-702e-48cd-bc43-83fbe83aead1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:43.130856 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-bac66bde-702e-48cd-bc43-83fbe83aead1 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:07:43.132214 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-bac66bde-702e-48cd-bc43-83fbe83aead1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:43.194104 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1357/2696] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:07:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:07:43.207532 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e5a42a-716c-4c77-8132-0af64c11007e admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:43.209247 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-15e5a42a-716c-4c77-8132-0af64c11007e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:43.270147 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e5a42a-716c-4c77-8132-0af64c11007e admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:43.273843 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478902e9-7558-481e-ac3e-470cb61c3e19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:43.274500 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478902e9-7558-481e-ac3e-470cb61c3e19 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:43.274781 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478902e9-7558-481e-ac3e-470cb61c3e19 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:43.275056 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478902e9-7558-481e-ac3e-470cb61c3e19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:43.286316 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478902e9-7558-481e-ac3e-470cb61c3e19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:43.287782 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478902e9-7558-481e-ac3e-470cb61c3e19 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:43.300572 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1340/2697] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:07:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:07:43.925088 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff755fc-3a38-4d45-9137-c6a1af190131 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:43.925623 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff755fc-3a38-4d45-9137-c6a1af190131 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:43.925755 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff755fc-3a38-4d45-9137-c6a1af190131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:43.952064 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-bff755fc-3a38-4d45-9137-c6a1af190131 None None] MFA Rules not processed for user `ad218437c31f443c8e009fffe6049edd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:07:43.953424 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-bff755fc-3a38-4d45-9137-c6a1af190131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:44.019162 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1358/2698] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:07:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 15:07:44.023715 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478902e9-7558-481e-ac3e-470cb61c3e19 service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:44.034376 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478902e9-7558-481e-ac3e-470cb61c3e19 service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:44.039133 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d628b9e-3ce0-4c5f-aea5-4d72bdb497bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:44.039869 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d628b9e-3ce0-4c5f-aea5-4d72bdb497bc admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:44.040163 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d628b9e-3ce0-4c5f-aea5-4d72bdb497bc admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:44.040444 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d628b9e-3ce0-4c5f-aea5-4d72bdb497bc admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:44.041242 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d628b9e-3ce0-4c5f-aea5-4d72bdb497bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:44.043168 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d628b9e-3ce0-4c5f-aea5-4d72bdb497bc admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:44.070139 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1341/2699] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:07:44 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:07:44.074569 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff755fc-3a38-4d45-9137-c6a1af190131 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:44.101412 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff755fc-3a38-4d45-9137-c6a1af190131 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:44.178391 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d741c879-071a-4d3f-8b42-06d86436b498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:44.179041 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741c879-071a-4d3f-8b42-06d86436b498 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:44.179397 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741c879-071a-4d3f-8b42-06d86436b498 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:44.179740 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741c879-071a-4d3f-8b42-06d86436b498 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:44.180571 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d741c879-071a-4d3f-8b42-06d86436b498 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:44.182770 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d741c879-071a-4d3f-8b42-06d86436b498 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:44.212827 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1359/2700] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:07:44 2022] POST /identity/v3/users => generated 293 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 15:07:44.214381 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d628b9e-3ce0-4c5f-aea5-4d72bdb497bc admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:44.232255 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d628b9e-3ce0-4c5f-aea5-4d72bdb497bc admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:44.238550 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd25584-7fe1-49d4-919d-292939268254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:44.239334 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd25584-7fe1-49d4-919d-292939268254 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:44.239688 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd25584-7fe1-49d4-919d-292939268254 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:44.240041 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd25584-7fe1-49d4-919d-292939268254 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:44.240909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd25584-7fe1-49d4-919d-292939268254 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:44.242586 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd25584-7fe1-49d4-919d-292939268254 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:44.254723 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1342/2701] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:07:44 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:07:44.258930 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d741c879-071a-4d3f-8b42-06d86436b498 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:44.271585 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d741c879-071a-4d3f-8b42-06d86436b498 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:44.277303 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a94da82-9c89-43d6-972f-a4a0533e0df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:44.278509 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a94da82-9c89-43d6-972f-a4a0533e0df1 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:44.278788 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a94da82-9c89-43d6-972f-a4a0533e0df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:44.279137 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a94da82-9c89-43d6-972f-a4a0533e0df1 admin admin] PATH_INFO: `/v3/projects/62b6099992ae45229ac1a6cab3eb1855/users/cf3229f9cce144a282bbc8ef1d59c5e5/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:44.306693 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a94da82-9c89-43d6-972f-a4a0533e0df1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62b6099992ae45229ac1a6cab3eb1855, user_id=cf3229f9cce144a282bbc8ef1d59c5e5, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:44.309947 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a94da82-9c89-43d6-972f-a4a0533e0df1 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:44.330869 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1360/2702] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:07:44 2022] PUT /identity/v3/projects/62b6099992ae45229ac1a6cab3eb1855/users/cf3229f9cce144a282bbc8ef1d59c5e5/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:07:44.334971 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd25584-7fe1-49d4-919d-292939268254 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:44.345608 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd25584-7fe1-49d4-919d-292939268254 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:44.407028 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af32210c-8433-47f9-bda5-f375907e9850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:44.409157 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32210c-8433-47f9-bda5-f375907e9850 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:44.409713 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32210c-8433-47f9-bda5-f375907e9850 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:44.410016 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32210c-8433-47f9-bda5-f375907e9850 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:44.410838 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af32210c-8433-47f9-bda5-f375907e9850 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:44.413597 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af32210c-8433-47f9-bda5-f375907e9850 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:44.426127 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1343/2703] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:07:44 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 06 15:07:44.430188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a94da82-9c89-43d6-972f-a4a0533e0df1 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:44.442997 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a94da82-9c89-43d6-972f-a4a0533e0df1 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:44.448144 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31e6975-21fe-4d04-af6e-1179e6fbd950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:44.449155 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e6975-21fe-4d04-af6e-1179e6fbd950 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:44.449431 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e6975-21fe-4d04-af6e-1179e6fbd950 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:44.449701 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e6975-21fe-4d04-af6e-1179e6fbd950 admin admin] PATH_INFO: `/v3/projects/62b6099992ae45229ac1a6cab3eb1855/users/cf3229f9cce144a282bbc8ef1d59c5e5/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:44.452314 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31e6975-21fe-4d04-af6e-1179e6fbd950 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62b6099992ae45229ac1a6cab3eb1855, user_id=cf3229f9cce144a282bbc8ef1d59c5e5, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:44.455246 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31e6975-21fe-4d04-af6e-1179e6fbd950 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:44.467668 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1361/2704] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:07:44 2022] PUT /identity/v3/projects/62b6099992ae45229ac1a6cab3eb1855/users/cf3229f9cce144a282bbc8ef1d59c5e5/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:07:45.193351 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1616b1ca-6a02-4d66-8290-8c96bee8a1a2 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:45.194225 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1616b1ca-6a02-4d66-8290-8c96bee8a1a2 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:45.194736 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1616b1ca-6a02-4d66-8290-8c96bee8a1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:45.230519 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-1616b1ca-6a02-4d66-8290-8c96bee8a1a2 None None] MFA Rules not processed for user `cf3229f9cce144a282bbc8ef1d59c5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:07:45.232821 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-1616b1ca-6a02-4d66-8290-8c96bee8a1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:45.307712 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1344/2705] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:07:45 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:07:45.319375 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31e6975-21fe-4d04-af6e-1179e6fbd950 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:45.321052 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e31e6975-21fe-4d04-af6e-1179e6fbd950 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:45.388100 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31e6975-21fe-4d04-af6e-1179e6fbd950 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:45.392385 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe3536-a8ad-4bc4-8c14-604dcf9b33c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:45.393298 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe3536-a8ad-4bc4-8c14-604dcf9b33c5 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:45.393588 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe3536-a8ad-4bc4-8c14-604dcf9b33c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:45.393871 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe3536-a8ad-4bc4-8c14-604dcf9b33c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:45.454860 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbe3536-a8ad-4bc4-8c14-604dcf9b33c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:45.456418 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbe3536-a8ad-4bc4-8c14-604dcf9b33c5 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:45.466325 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1362/2706] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:07:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:07:46.852785 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1616b1ca-6a02-4d66-8290-8c96bee8a1a2 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:46.854259 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-1616b1ca-6a02-4d66-8290-8c96bee8a1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:46.867394 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1616b1ca-6a02-4d66-8290-8c96bee8a1a2 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:46.870915 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87b76a-4638-4366-b9b5-76fb5e9f9332 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:46.871557 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87b76a-4638-4366-b9b5-76fb5e9f9332 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:46.871841 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87b76a-4638-4366-b9b5-76fb5e9f9332 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:46.872117 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87b76a-4638-4366-b9b5-76fb5e9f9332 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:46.881090 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a87b76a-4638-4366-b9b5-76fb5e9f9332 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:46.882563 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a87b76a-4638-4366-b9b5-76fb5e9f9332 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:07:46.892353 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1345/2707] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:07:46 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:07:59.868512 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe3536-a8ad-4bc4-8c14-604dcf9b33c5 service cinder] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:07:59.924349 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe3536-a8ad-4bc4-8c14-604dcf9b33c5 service cinder] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:07:59.927890 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56fc27b-c080-4560-bb9a-881a0221e25c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:07:59.928644 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56fc27b-c080-4560-bb9a-881a0221e25c service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:07:59.928930 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56fc27b-c080-4560-bb9a-881a0221e25c service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:07:59.929255 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56fc27b-c080-4560-bb9a-881a0221e25c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:07:59.931054 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f56fc27b-c080-4560-bb9a-881a0221e25c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:07:59.994956 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56fc27b-c080-4560-bb9a-881a0221e25c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:07:59.996420 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56fc27b-c080-4560-bb9a-881a0221e25c service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:08:00.007026 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1363/2708] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:07:59 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:08:00.038588 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87b76a-4638-4366-b9b5-76fb5e9f9332 service cinder] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:08:00.106108 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87b76a-4638-4366-b9b5-76fb5e9f9332 service cinder] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:08:00.112088 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88a062c-616d-435e-a2f6-a4cfdec96e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:08:00.112992 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a062c-616d-435e-a2f6-a4cfdec96e84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:08:00.113291 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a062c-616d-435e-a2f6-a4cfdec96e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:08:00.113567 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a062c-616d-435e-a2f6-a4cfdec96e84 admin admin] PATH_INFO: `/v3/users/ad218437c31f443c8e009fffe6049edd` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:08:00.114767 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88a062c-616d-435e-a2f6-a4cfdec96e84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad218437c31f443c8e009fffe6049edd)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:08:00.116737 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88a062c-616d-435e-a2f6-a4cfdec96e84 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:08:00.165165 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d88a062c-616d-435e-a2f6-a4cfdec96e84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad218437c31f443c8e009fffe6049edd'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:08:00.166011 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d88a062c-616d-435e-a2f6-a4cfdec96e84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad218437c31f443c8e009fffe6049edd'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:08:00.173967 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d88a062c-616d-435e-a2f6-a4cfdec96e84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad218437c31f443c8e009fffe6049edd'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:08:00.182446 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d88a062c-616d-435e-a2f6-a4cfdec96e84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad218437c31f443c8e009fffe6049edd'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:08:00.183108 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d88a062c-616d-435e-a2f6-a4cfdec96e84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad218437c31f443c8e009fffe6049edd'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:08:00.200812 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1346/2709] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:08:00 2022] DELETE /identity/v3/users/ad218437c31f443c8e009fffe6049edd => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:08:00.374903 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56fc27b-c080-4560-bb9a-881a0221e25c service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:08:00.404406 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56fc27b-c080-4560-bb9a-881a0221e25c service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:08:00.470172 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb608ea1-90b6-436e-b903-6b6b46515a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:08:00.471045 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb608ea1-90b6-436e-b903-6b6b46515a50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:08:00.471320 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb608ea1-90b6-436e-b903-6b6b46515a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:08:00.471592 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb608ea1-90b6-436e-b903-6b6b46515a50 admin admin] PATH_INFO: `/v3/projects/ad326e01031b493cb0c63533d25a2ec3` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:08:00.472821 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb608ea1-90b6-436e-b903-6b6b46515a50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad326e01031b493cb0c63533d25a2ec3)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:08:00.474724 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb608ea1-90b6-436e-b903-6b6b46515a50 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:08:00.522031 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-cb608ea1-90b6-436e-b903-6b6b46515a50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad326e01031b493cb0c63533d25a2ec3', 'request_id': 'req-cb608ea1-90b6-436e-b903-6b6b46515a50'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:08:00.527215 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-cb608ea1-90b6-436e-b903-6b6b46515a50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad326e01031b493cb0c63533d25a2ec3', 'request_id': 'req-cb608ea1-90b6-436e-b903-6b6b46515a50'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:08:00.536178 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-cb608ea1-90b6-436e-b903-6b6b46515a50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad326e01031b493cb0c63533d25a2ec3', 'request_id': 'req-cb608ea1-90b6-436e-b903-6b6b46515a50'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:08:00.547105 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-cb608ea1-90b6-436e-b903-6b6b46515a50 admin admin] The token cache is being invalidate because project ad326e01031b493cb0c63533d25a2ec3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:08:00.547397 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-cb608ea1-90b6-436e-b903-6b6b46515a50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:08:00.548669 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-cb608ea1-90b6-436e-b903-6b6b46515a50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:08:00.550237 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1364/2710] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:08:00 2022] DELETE /identity/v3/projects/ad326e01031b493cb0c63533d25a2ec3 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:08:00.554282 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88a062c-616d-435e-a2f6-a4cfdec96e84 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:08:00.555707 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d88a062c-616d-435e-a2f6-a4cfdec96e84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:08:00.568693 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88a062c-616d-435e-a2f6-a4cfdec96e84 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:08:00.625243 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0114a0d3-1418-4aa3-bb7f-dde3136573ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:08:00.626128 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114a0d3-1418-4aa3-bb7f-dde3136573ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:08:00.626564 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114a0d3-1418-4aa3-bb7f-dde3136573ae admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:08:00.626845 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114a0d3-1418-4aa3-bb7f-dde3136573ae admin admin] PATH_INFO: `/v3/users/cf3229f9cce144a282bbc8ef1d59c5e5` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:08:00.628047 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0114a0d3-1418-4aa3-bb7f-dde3136573ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf3229f9cce144a282bbc8ef1d59c5e5)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:08:00.630006 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0114a0d3-1418-4aa3-bb7f-dde3136573ae admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:08:00.679487 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-0114a0d3-1418-4aa3-bb7f-dde3136573ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf3229f9cce144a282bbc8ef1d59c5e5'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:08:00.680151 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-0114a0d3-1418-4aa3-bb7f-dde3136573ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf3229f9cce144a282bbc8ef1d59c5e5'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:08:00.688755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-0114a0d3-1418-4aa3-bb7f-dde3136573ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf3229f9cce144a282bbc8ef1d59c5e5'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:08:00.696045 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-0114a0d3-1418-4aa3-bb7f-dde3136573ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf3229f9cce144a282bbc8ef1d59c5e5'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:08:00.696755 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-0114a0d3-1418-4aa3-bb7f-dde3136573ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf3229f9cce144a282bbc8ef1d59c5e5'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:08:00.715204 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1347/2711] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:08:00 2022] DELETE /identity/v3/users/cf3229f9cce144a282bbc8ef1d59c5e5 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 15:08:00.894532 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb608ea1-90b6-436e-b903-6b6b46515a50 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:08:00.896152 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-cb608ea1-90b6-436e-b903-6b6b46515a50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:08:00.911423 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb608ea1-90b6-436e-b903-6b6b46515a50 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:08:00.979012 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c89408-9f0c-46c4-b559-1a60a31e44e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:08:00.979837 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c89408-9f0c-46c4-b559-1a60a31e44e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:08:00.980117 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c89408-9f0c-46c4-b559-1a60a31e44e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:08:00.980534 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c89408-9f0c-46c4-b559-1a60a31e44e7 admin admin] PATH_INFO: `/v3/projects/62b6099992ae45229ac1a6cab3eb1855` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:08:00.981782 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c89408-9f0c-46c4-b559-1a60a31e44e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62b6099992ae45229ac1a6cab3eb1855)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:08:00.983748 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c89408-9f0c-46c4-b559-1a60a31e44e7 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:08:01.032453 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-49c89408-9f0c-46c4-b559-1a60a31e44e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62b6099992ae45229ac1a6cab3eb1855', 'request_id': 'req-49c89408-9f0c-46c4-b559-1a60a31e44e7'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:08:01.038020 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-49c89408-9f0c-46c4-b559-1a60a31e44e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62b6099992ae45229ac1a6cab3eb1855', 'request_id': 'req-49c89408-9f0c-46c4-b559-1a60a31e44e7'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:08:01.046797 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-49c89408-9f0c-46c4-b559-1a60a31e44e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62b6099992ae45229ac1a6cab3eb1855', 'request_id': 'req-49c89408-9f0c-46c4-b559-1a60a31e44e7'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:08:01.059656 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-49c89408-9f0c-46c4-b559-1a60a31e44e7 admin admin] The token cache is being invalidate because project 62b6099992ae45229ac1a6cab3eb1855 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:08:01.060112 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-49c89408-9f0c-46c4-b559-1a60a31e44e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:08:01.060632 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-49c89408-9f0c-46c4-b559-1a60a31e44e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:08:01.062298 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1365/2712] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:08:00 2022] DELETE /identity/v3/projects/62b6099992ae45229ac1a6cab3eb1855 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 15:08:01.069028 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66cb5f-93a7-473d-8738-15f1e5b6e80e None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:08:01.069593 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66cb5f-93a7-473d-8738-15f1e5b6e80e None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:08:01.069715 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66cb5f-93a7-473d-8738-15f1e5b6e80e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:08:01.089225 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-8d66cb5f-93a7-473d-8738-15f1e5b6e80e None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:08:01.090435 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-8d66cb5f-93a7-473d-8738-15f1e5b6e80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:08:01.160026 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1348/2713] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:08:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:08:01.163424 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e3e72-498b-4db3-acd5-b59a05c79456 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:08:01.164435 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e3e72-498b-4db3-acd5-b59a05c79456 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:08:01.164818 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e3e72-498b-4db3-acd5-b59a05c79456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:08:01.183604 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-dd1e3e72-498b-4db3-acd5-b59a05c79456 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:08:01.184867 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-dd1e3e72-498b-4db3-acd5-b59a05c79456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:08:01.193300 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1366/2714] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:08:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:08:01.198080 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d66cb5f-93a7-473d-8738-15f1e5b6e80e None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:08:01.207972 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d66cb5f-93a7-473d-8738-15f1e5b6e80e None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:08:01.212516 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c71c49-5253-4a44-b0f4-c2677567f886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:08:01.213222 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c71c49-5253-4a44-b0f4-c2677567f886 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:08:01.213507 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c71c49-5253-4a44-b0f4-c2677567f886 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:08:01.213778 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c71c49-5253-4a44-b0f4-c2677567f886 admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:08:01.214494 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c71c49-5253-4a44-b0f4-c2677567f886 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 15:08:01.214799 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c71c49-5253-4a44-b0f4-c2677567f886 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:08:01.216526 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c71c49-5253-4a44-b0f4-c2677567f886 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:08:01.229416 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1349/2715] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 15:08:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 15:08:01.234065 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e3e72-498b-4db3-acd5-b59a05c79456 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:08:01.243514 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e3e72-498b-4db3-acd5-b59a05c79456 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:08:01.248287 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83a5b9-a9a2-4427-b2b2-a25d33dab770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:08:01.248973 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83a5b9-a9a2-4427-b2b2-a25d33dab770 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:08:01.249263 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83a5b9-a9a2-4427-b2b2-a25d33dab770 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:08:01.249541 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83a5b9-a9a2-4427-b2b2-a25d33dab770 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:08:01.250237 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d83a5b9-a9a2-4427-b2b2-a25d33dab770 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:08:01.252872 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d83a5b9-a9a2-4427-b2b2-a25d33dab770 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:08:01.278806 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1367/2716] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:08:01 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:08:01.283013 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c71c49-5253-4a44-b0f4-c2677567f886 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:08:01.292883 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c71c49-5253-4a44-b0f4-c2677567f886 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:08:01.352388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68137b04-196f-4ea7-b225-ac6e729a1d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:08:01.353091 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68137b04-196f-4ea7-b225-ac6e729a1d82 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:08:01.353389 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68137b04-196f-4ea7-b225-ac6e729a1d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:08:01.353654 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68137b04-196f-4ea7-b225-ac6e729a1d82 admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:08:01.354334 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68137b04-196f-4ea7-b225-ac6e729a1d82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:08:01.356295 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68137b04-196f-4ea7-b225-ac6e729a1d82 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:08:01.380979 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1350/2717] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:08:01 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:08:01.382435 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83a5b9-a9a2-4427-b2b2-a25d33dab770 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:08:01.392822 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83a5b9-a9a2-4427-b2b2-a25d33dab770 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:08:01.397451 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2511efe1-bf76-416b-b7f6-6335cb442858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:08:01.398108 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511efe1-bf76-416b-b7f6-6335cb442858 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:08:01.398391 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511efe1-bf76-416b-b7f6-6335cb442858 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:08:01.398666 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511efe1-bf76-416b-b7f6-6335cb442858 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:08:01.399405 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2511efe1-bf76-416b-b7f6-6335cb442858 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:08:01.401018 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2511efe1-bf76-416b-b7f6-6335cb442858 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:08:01.412010 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1368/2718] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:08:01.416138 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68137b04-196f-4ea7-b225-ac6e729a1d82 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:08:01.426727 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68137b04-196f-4ea7-b225-ac6e729a1d82 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:08:01.431652 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c5288-1c3b-4867-bbee-1c209ea07092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:08:01.432318 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c5288-1c3b-4867-bbee-1c209ea07092 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:08:01.432616 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c5288-1c3b-4867-bbee-1c209ea07092 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:08:01.432907 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c5288-1c3b-4867-bbee-1c209ea07092 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:08:01.433633 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c5288-1c3b-4867-bbee-1c209ea07092 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:08:01.435257 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c5288-1c3b-4867-bbee-1c209ea07092 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:08:01.446487 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1351/2719] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:08:01.450690 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2511efe1-bf76-416b-b7f6-6335cb442858 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:08:01.462570 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2511efe1-bf76-416b-b7f6-6335cb442858 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:08:01.467873 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7fa5d3-7936-4317-84c9-a9a3bd4a5903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:08:01.468933 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fa5d3-7936-4317-84c9-a9a3bd4a5903 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:08:01.469256 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fa5d3-7936-4317-84c9-a9a3bd4a5903 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:08:01.469561 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fa5d3-7936-4317-84c9-a9a3bd4a5903 admin admin] PATH_INFO: `/v3/projects/7cec864a4e3948a0946d0ce68c37865b/users/b5ca5018dd9944ee94b06c2d15767054/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:08:01.484375 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7fa5d3-7936-4317-84c9-a9a3bd4a5903 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cec864a4e3948a0946d0ce68c37865b, user_id=b5ca5018dd9944ee94b06c2d15767054, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:08:01.487955 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7fa5d3-7936-4317-84c9-a9a3bd4a5903 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:08:01.502781 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1369/2720] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:08:01 2022] PUT /identity/v3/projects/7cec864a4e3948a0946d0ce68c37865b/users/b5ca5018dd9944ee94b06c2d15767054/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 15:08:01.512994 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d470763-47cb-4a2d-8e47-a5b9a8ad37d4 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:08:01.513468 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d470763-47cb-4a2d-8e47-a5b9a8ad37d4 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:08:01.514472 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d470763-47cb-4a2d-8e47-a5b9a8ad37d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:08:01.541992 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-3d470763-47cb-4a2d-8e47-a5b9a8ad37d4 None None] MFA Rules not processed for user `b5ca5018dd9944ee94b06c2d15767054`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:08:01.543233 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-3d470763-47cb-4a2d-8e47-a5b9a8ad37d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:08:01.606426 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1352/2721] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:08:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:08:01.617804 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7fa5d3-7936-4317-84c9-a9a3bd4a5903 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:08:01.619505 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-cc7fa5d3-7936-4317-84c9-a9a3bd4a5903 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:08:01.678128 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7fa5d3-7936-4317-84c9-a9a3bd4a5903 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:08:01.682088 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a17d96-8bcd-43d6-a800-8168de7eb57d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9be93f9ff7e46babc8ee87b50b9c5eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:08:01.682749 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a17d96-8bcd-43d6-a800-8168de7eb57d service glance] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:08:01.683033 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a17d96-8bcd-43d6-a800-8168de7eb57d service glance] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:08:01.683317 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a17d96-8bcd-43d6-a800-8168de7eb57d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:08:01.692862 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a17d96-8bcd-43d6-a800-8168de7eb57d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:08:01.694295 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a17d96-8bcd-43d6-a800-8168de7eb57d service glance] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:08:01.703087 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1370/2722] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:08:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:08:21.759116 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d470763-47cb-4a2d-8e47-a5b9a8ad37d4 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:08:21.760508 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-3d470763-47cb-4a2d-8e47-a5b9a8ad37d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:08:21.816744 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d470763-47cb-4a2d-8e47-a5b9a8ad37d4 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:08:21.820225 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadccb8e-c3ec-4d94-80c7-bde99d0a34e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:08:21.820902 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadccb8e-c3ec-4d94-80c7-bde99d0a34e3 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:08:21.821192 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadccb8e-c3ec-4d94-80c7-bde99d0a34e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:08:21.821474 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadccb8e-c3ec-4d94-80c7-bde99d0a34e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:08:21.823226 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-dadccb8e-c3ec-4d94-80c7-bde99d0a34e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:08:21.892015 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadccb8e-c3ec-4d94-80c7-bde99d0a34e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:08:21.894104 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadccb8e-c3ec-4d94-80c7-bde99d0a34e3 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:08:21.905127 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1353/2723] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:08:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 15:08:23.026838 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a17d96-8bcd-43d6-a800-8168de7eb57d service glance] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:08:23.028313 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-91a17d96-8bcd-43d6-a800-8168de7eb57d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:08:23.040821 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a17d96-8bcd-43d6-a800-8168de7eb57d service glance] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:08:23.044440 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3b39e8-fa7a-4cc0-8293-c04d1f77bd89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:08:23.045107 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b39e8-fa7a-4cc0-8293-c04d1f77bd89 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:08:23.045397 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b39e8-fa7a-4cc0-8293-c04d1f77bd89 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:08:23.045676 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b39e8-fa7a-4cc0-8293-c04d1f77bd89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:08:23.047354 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-9e3b39e8-fa7a-4cc0-8293-c04d1f77bd89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:08:23.061522 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3b39e8-fa7a-4cc0-8293-c04d1f77bd89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:08:23.062871 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3b39e8-fa7a-4cc0-8293-c04d1f77bd89 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:08:23.073778 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1371/2724] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:08:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:08:23.487836 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadccb8e-c3ec-4d94-80c7-bde99d0a34e3 service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:08:23.554885 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadccb8e-c3ec-4d94-80c7-bde99d0a34e3 service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:08:23.559492 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b203b-b8e1-4e6a-838e-2505453081ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:08:23.560377 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b203b-b8e1-4e6a-838e-2505453081ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:08:23.560697 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b203b-b8e1-4e6a-838e-2505453081ca admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:08:23.560965 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b203b-b8e1-4e6a-838e-2505453081ca admin admin] PATH_INFO: `/v3/users/b5ca5018dd9944ee94b06c2d15767054` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:08:23.562171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654b203b-b8e1-4e6a-838e-2505453081ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5ca5018dd9944ee94b06c2d15767054)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:08:23.564688 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654b203b-b8e1-4e6a-838e-2505453081ca admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:08:23.613550 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-654b203b-b8e1-4e6a-838e-2505453081ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5ca5018dd9944ee94b06c2d15767054'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:08:23.614205 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-654b203b-b8e1-4e6a-838e-2505453081ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5ca5018dd9944ee94b06c2d15767054'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:08:23.621476 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-654b203b-b8e1-4e6a-838e-2505453081ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5ca5018dd9944ee94b06c2d15767054'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:08:23.630248 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-654b203b-b8e1-4e6a-838e-2505453081ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5ca5018dd9944ee94b06c2d15767054'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:08:23.631036 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-654b203b-b8e1-4e6a-838e-2505453081ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5ca5018dd9944ee94b06c2d15767054'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:08:23.650350 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1354/2725] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:08:23 2022] DELETE /identity/v3/users/b5ca5018dd9944ee94b06c2d15767054 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:08:23.655244 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c74b88-26e3-4e72-b8d3-3cbf11924f08 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:08:23.655535 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c74b88-26e3-4e72-b8d3-3cbf11924f08 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:08:23.655836 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c74b88-26e3-4e72-b8d3-3cbf11924f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:08:23.673304 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-a2c74b88-26e3-4e72-b8d3-3cbf11924f08 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:08:23.674968 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-a2c74b88-26e3-4e72-b8d3-3cbf11924f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:08:23.741837 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1372/2726] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:08:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:08:23.753671 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b203b-b8e1-4e6a-838e-2505453081ca admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:08:23.755274 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-654b203b-b8e1-4e6a-838e-2505453081ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:08:23.818945 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b203b-b8e1-4e6a-838e-2505453081ca admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:08:23.825260 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f685d62-5cbc-4ee3-969c-570bf894f726 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:08:23.828291 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f685d62-5cbc-4ee3-969c-570bf894f726 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:08:23.828291 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f685d62-5cbc-4ee3-969c-570bf894f726 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:08:23.828291 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f685d62-5cbc-4ee3-969c-570bf894f726 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:08:23.837704 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f685d62-5cbc-4ee3-969c-570bf894f726 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:08:23.839145 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f685d62-5cbc-4ee3-969c-570bf894f726 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:08:23.849053 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1355/2727] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:08:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:08:24.370017 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c74b88-26e3-4e72-b8d3-3cbf11924f08 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:08:24.371601 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-a2c74b88-26e3-4e72-b8d3-3cbf11924f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:08:24.384132 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c74b88-26e3-4e72-b8d3-3cbf11924f08 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:08:24.389083 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24c981-b516-4421-a934-e706683e455e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:08:24.389956 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24c981-b516-4421-a934-e706683e455e admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:08:24.390242 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24c981-b516-4421-a934-e706683e455e admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:08:24.390522 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24c981-b516-4421-a934-e706683e455e admin admin] PATH_INFO: `/v3/projects/7cec864a4e3948a0946d0ce68c37865b` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:08:24.391709 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa24c981-b516-4421-a934-e706683e455e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cec864a4e3948a0946d0ce68c37865b)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:08:24.393853 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa24c981-b516-4421-a934-e706683e455e admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:08:24.455027 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-aa24c981-b516-4421-a934-e706683e455e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cec864a4e3948a0946d0ce68c37865b', 'request_id': 'req-aa24c981-b516-4421-a934-e706683e455e'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:08:24.460908 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-aa24c981-b516-4421-a934-e706683e455e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cec864a4e3948a0946d0ce68c37865b', 'request_id': 'req-aa24c981-b516-4421-a934-e706683e455e'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:08:24.469855 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-aa24c981-b516-4421-a934-e706683e455e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cec864a4e3948a0946d0ce68c37865b', 'request_id': 'req-aa24c981-b516-4421-a934-e706683e455e'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:08:24.492147 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-aa24c981-b516-4421-a934-e706683e455e admin admin] The token cache is being invalidate because project 7cec864a4e3948a0946d0ce68c37865b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:08:24.492448 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-aa24c981-b516-4421-a934-e706683e455e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:08:24.493238 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-aa24c981-b516-4421-a934-e706683e455e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:08:24.494883 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1373/2728] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:08:24 2022] DELETE /identity/v3/projects/7cec864a4e3948a0946d0ce68c37865b => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:08:28.906300 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f685d62-5cbc-4ee3-969c-570bf894f726 service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:08:28.915715 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f685d62-5cbc-4ee3-969c-570bf894f726 service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:08:28.967400 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311dc84a-2a73-406b-a1fd-f51158ef2b08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:08:28.968071 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311dc84a-2a73-406b-a1fd-f51158ef2b08 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:08:28.968360 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311dc84a-2a73-406b-a1fd-f51158ef2b08 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:08:28.968668 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311dc84a-2a73-406b-a1fd-f51158ef2b08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:08:28.970652 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-311dc84a-2a73-406b-a1fd-f51158ef2b08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:08:29.036831 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311dc84a-2a73-406b-a1fd-f51158ef2b08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:08:29.038275 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311dc84a-2a73-406b-a1fd-f51158ef2b08 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:08:29.048983 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1356/2729] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:08:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:08:32.380298 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24c981-b516-4421-a934-e706683e455e admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:08:32.382217 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-aa24c981-b516-4421-a934-e706683e455e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:08:32.397562 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24c981-b516-4421-a934-e706683e455e admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:08:32.402213 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cfa728-aa2c-4e87-a52f-d681bfafd74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:08:32.403094 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cfa728-aa2c-4e87-a52f-d681bfafd74b admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:08:32.403378 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cfa728-aa2c-4e87-a52f-d681bfafd74b admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:08:32.403923 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cfa728-aa2c-4e87-a52f-d681bfafd74b admin admin] PATH_INFO: `/v3/users/02b07e776b2c4bfba6eabb15999aec35` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:08:32.418564 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cfa728-aa2c-4e87-a52f-d681bfafd74b admin admin] RBAC: Authorizing `identity:delete_user(user_id=02b07e776b2c4bfba6eabb15999aec35)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:08:32.420505 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cfa728-aa2c-4e87-a52f-d681bfafd74b admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:08:32.468638 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-e0cfa728-aa2c-4e87-a52f-d681bfafd74b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02b07e776b2c4bfba6eabb15999aec35'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:08:32.476885 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-e0cfa728-aa2c-4e87-a52f-d681bfafd74b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02b07e776b2c4bfba6eabb15999aec35'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:08:32.483752 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-e0cfa728-aa2c-4e87-a52f-d681bfafd74b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02b07e776b2c4bfba6eabb15999aec35'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:08:32.484424 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-e0cfa728-aa2c-4e87-a52f-d681bfafd74b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02b07e776b2c4bfba6eabb15999aec35'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:08:32.485060 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-e0cfa728-aa2c-4e87-a52f-d681bfafd74b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02b07e776b2c4bfba6eabb15999aec35'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:08:32.505788 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1374/2730] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:08:32 2022] DELETE /identity/v3/users/02b07e776b2c4bfba6eabb15999aec35 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:08:32.682157 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311dc84a-2a73-406b-a1fd-f51158ef2b08 service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:08:32.693339 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311dc84a-2a73-406b-a1fd-f51158ef2b08 service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:08:32.760345 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3df74-ba58-4247-97d3-0b53b6b83608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:08:32.761264 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3df74-ba58-4247-97d3-0b53b6b83608 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:08:32.761544 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3df74-ba58-4247-97d3-0b53b6b83608 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:08:32.761816 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3df74-ba58-4247-97d3-0b53b6b83608 admin admin] PATH_INFO: `/v3/projects/04793e5505aa4d32ab3efed2892ea2b4` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:08:32.773455 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff3df74-ba58-4247-97d3-0b53b6b83608 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04793e5505aa4d32ab3efed2892ea2b4)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:08:32.775480 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff3df74-ba58-4247-97d3-0b53b6b83608 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:08:32.826289 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-6ff3df74-ba58-4247-97d3-0b53b6b83608 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04793e5505aa4d32ab3efed2892ea2b4', 'request_id': 'req-6ff3df74-ba58-4247-97d3-0b53b6b83608'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:08:32.832555 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-6ff3df74-ba58-4247-97d3-0b53b6b83608 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04793e5505aa4d32ab3efed2892ea2b4', 'request_id': 'req-6ff3df74-ba58-4247-97d3-0b53b6b83608'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:08:32.840940 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-6ff3df74-ba58-4247-97d3-0b53b6b83608 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04793e5505aa4d32ab3efed2892ea2b4', 'request_id': 'req-6ff3df74-ba58-4247-97d3-0b53b6b83608'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:08:32.854215 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-6ff3df74-ba58-4247-97d3-0b53b6b83608 admin admin] The token cache is being invalidate because project 04793e5505aa4d32ab3efed2892ea2b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:08:32.854509 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-6ff3df74-ba58-4247-97d3-0b53b6b83608 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:08:32.855166 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-6ff3df74-ba58-4247-97d3-0b53b6b83608 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:08:32.856773 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1357/2731] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:08:32 2022] DELETE /identity/v3/projects/04793e5505aa4d32ab3efed2892ea2b4 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:08:32.862568 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cbf7f-80f6-4e50-a399-02ce32223ef9 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:08:32.862848 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cbf7f-80f6-4e50-a399-02ce32223ef9 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:08:32.863127 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cbf7f-80f6-4e50-a399-02ce32223ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:08:32.881891 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-a07cbf7f-80f6-4e50-a399-02ce32223ef9 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:08:32.883107 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-a07cbf7f-80f6-4e50-a399-02ce32223ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:08:32.945798 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1375/2732] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:08:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:08:32.955942 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae308b56-a537-44e2-a2ea-c8ead5452d7b None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:08:32.956318 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae308b56-a537-44e2-a2ea-c8ead5452d7b None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:08:32.956669 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae308b56-a537-44e2-a2ea-c8ead5452d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:08:32.976116 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-ae308b56-a537-44e2-a2ea-c8ead5452d7b None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:08:32.977347 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ae308b56-a537-44e2-a2ea-c8ead5452d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:08:32.985248 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1358/2733] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:08:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 15:08:32.989775 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07cbf7f-80f6-4e50-a399-02ce32223ef9 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:08:32.999365 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07cbf7f-80f6-4e50-a399-02ce32223ef9 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:08:33.004579 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc40ff8-3847-4dbf-b716-bd3de04f5479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:08:33.005271 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc40ff8-3847-4dbf-b716-bd3de04f5479 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:08:33.005547 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc40ff8-3847-4dbf-b716-bd3de04f5479 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:08:33.005825 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc40ff8-3847-4dbf-b716-bd3de04f5479 admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:08:33.006551 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc40ff8-3847-4dbf-b716-bd3de04f5479 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 15:08:33.006854 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc40ff8-3847-4dbf-b716-bd3de04f5479 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:08:33.008440 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc40ff8-3847-4dbf-b716-bd3de04f5479 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:08:33.023793 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1376/2734] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 15:08:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 15:08:33.029486 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae308b56-a537-44e2-a2ea-c8ead5452d7b None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:08:33.040850 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae308b56-a537-44e2-a2ea-c8ead5452d7b None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:08:33.045560 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782f1ded-9dc8-4963-b438-7b3e9602b33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:08:33.046222 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f1ded-9dc8-4963-b438-7b3e9602b33a admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:08:33.046506 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f1ded-9dc8-4963-b438-7b3e9602b33a admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:08:33.046780 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f1ded-9dc8-4963-b438-7b3e9602b33a admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:08:33.047463 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782f1ded-9dc8-4963-b438-7b3e9602b33a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:08:33.049412 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782f1ded-9dc8-4963-b438-7b3e9602b33a admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:08:33.076413 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1359/2735] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:08:33 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:08:33.081500 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc40ff8-3847-4dbf-b716-bd3de04f5479 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:08:33.091781 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc40ff8-3847-4dbf-b716-bd3de04f5479 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:08:33.151352 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ea4f96-70c9-4537-92cf-10098e5980c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:08:33.152018 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea4f96-70c9-4537-92cf-10098e5980c8 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:08:33.152301 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea4f96-70c9-4537-92cf-10098e5980c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:08:33.152577 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea4f96-70c9-4537-92cf-10098e5980c8 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:08:33.153323 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ea4f96-70c9-4537-92cf-10098e5980c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:08:33.155519 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ea4f96-70c9-4537-92cf-10098e5980c8 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:08:33.178601 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1377/2736] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:08:33 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:08:33.182670 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782f1ded-9dc8-4963-b438-7b3e9602b33a admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:08:33.193462 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782f1ded-9dc8-4963-b438-7b3e9602b33a admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:08:33.198051 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576eafe-8b58-4039-b8d6-c22245a86921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:08:33.198699 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576eafe-8b58-4039-b8d6-c22245a86921 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:08:33.198980 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576eafe-8b58-4039-b8d6-c22245a86921 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:08:33.199253 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576eafe-8b58-4039-b8d6-c22245a86921 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:08:33.199973 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2576eafe-8b58-4039-b8d6-c22245a86921 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:08:33.201591 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2576eafe-8b58-4039-b8d6-c22245a86921 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:08:33.212659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1360/2737] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:08:33.216825 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ea4f96-70c9-4537-92cf-10098e5980c8 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:08:33.227292 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ea4f96-70c9-4537-92cf-10098e5980c8 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:08:33.231969 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f03cbf-5f96-4e55-81a3-0a3300de622e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:08:33.232671 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f03cbf-5f96-4e55-81a3-0a3300de622e admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:08:33.232961 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f03cbf-5f96-4e55-81a3-0a3300de622e admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:08:33.233243 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f03cbf-5f96-4e55-81a3-0a3300de622e admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:08:33.233991 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f03cbf-5f96-4e55-81a3-0a3300de622e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:08:33.235578 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f03cbf-5f96-4e55-81a3-0a3300de622e admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:08:33.245486 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1378/2738] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:08:33.249895 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576eafe-8b58-4039-b8d6-c22245a86921 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:08:33.260278 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576eafe-8b58-4039-b8d6-c22245a86921 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:08:33.264935 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd445654-f815-40c6-bcef-fcfff461f048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:08:33.265804 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd445654-f815-40c6-bcef-fcfff461f048 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:08:33.266089 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd445654-f815-40c6-bcef-fcfff461f048 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:08:33.266367 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd445654-f815-40c6-bcef-fcfff461f048 admin admin] PATH_INFO: `/v3/projects/6f2937968fc04cfaa3dab933e994a975/users/90accf4926bf452d8ff6e79a1c784911/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:08:33.281854 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd445654-f815-40c6-bcef-fcfff461f048 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f2937968fc04cfaa3dab933e994a975, user_id=90accf4926bf452d8ff6e79a1c784911, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:08:33.285039 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd445654-f815-40c6-bcef-fcfff461f048 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:08:33.297441 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1361/2739] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:08:33 2022] PUT /identity/v3/projects/6f2937968fc04cfaa3dab933e994a975/users/90accf4926bf452d8ff6e79a1c784911/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 15:08:33.303182 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef4de3-6755-4c3f-b3e7-17810b54a033 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:08:33.303484 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef4de3-6755-4c3f-b3e7-17810b54a033 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:08:33.303766 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef4de3-6755-4c3f-b3e7-17810b54a033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:08:33.328259 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-9bef4de3-6755-4c3f-b3e7-17810b54a033 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:08:33.329397 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-9bef4de3-6755-4c3f-b3e7-17810b54a033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:08:33.389940 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1379/2740] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:08:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:08:33.400126 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd445654-f815-40c6-bcef-fcfff461f048 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:08:33.401830 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-dd445654-f815-40c6-bcef-fcfff461f048 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:08:33.457216 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd445654-f815-40c6-bcef-fcfff461f048 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:08:33.460819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee86371-dc2a-49c8-b6de-445c036a38e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:08:33.461452 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee86371-dc2a-49c8-b6de-445c036a38e8 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:08:33.461753 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee86371-dc2a-49c8-b6de-445c036a38e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:08:33.462026 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee86371-dc2a-49c8-b6de-445c036a38e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:08:33.471126 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee86371-dc2a-49c8-b6de-445c036a38e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:08:33.472539 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee86371-dc2a-49c8-b6de-445c036a38e8 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:08:33.481630 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1362/2741] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:08:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:08:38.800487 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2620aa76-da59-4280-a828-baa00fa5ee9e None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:08:38.801821 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2620aa76-da59-4280-a828-baa00fa5ee9e None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:08:38.805276 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2620aa76-da59-4280-a828-baa00fa5ee9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:08:38.865971 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-2620aa76-da59-4280-a828-baa00fa5ee9e None None] MFA Rules not processed for user `90accf4926bf452d8ff6e79a1c784911`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:08:38.867552 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-2620aa76-da59-4280-a828-baa00fa5ee9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:08:38.973632 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1380/2742] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:08:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:08:38.990071 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee86371-dc2a-49c8-b6de-445c036a38e8 service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:08:38.991876 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-6ee86371-dc2a-49c8-b6de-445c036a38e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:08:39.094075 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee86371-dc2a-49c8-b6de-445c036a38e8 service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:08:39.101102 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a80d0-1580-4a3f-a4d4-74e2d7c21133 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:08:39.101691 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a80d0-1580-4a3f-a4d4-74e2d7c21133 service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:08:39.101969 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a80d0-1580-4a3f-a4d4-74e2d7c21133 service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:08:39.102241 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a80d0-1580-4a3f-a4d4-74e2d7c21133 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:08:39.112938 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4a80d0-1580-4a3f-a4d4-74e2d7c21133 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:08:39.114749 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4a80d0-1580-4a3f-a4d4-74e2d7c21133 service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:08:39.127464 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1363/2743] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:08:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:08:40.928756 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7830197-9ec0-4a0b-bec4-c0a1c10aeed9 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:08:40.929350 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7830197-9ec0-4a0b-bec4-c0a1c10aeed9 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:08:40.929350 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7830197-9ec0-4a0b-bec4-c0a1c10aeed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:08:40.947778 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-f7830197-9ec0-4a0b-bec4-c0a1c10aeed9 None None] MFA Rules not processed for user `90accf4926bf452d8ff6e79a1c784911`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:08:40.949029 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f7830197-9ec0-4a0b-bec4-c0a1c10aeed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:08:40.956136 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1381/2744] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:08:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:08:41.715664 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a80d0-1580-4a3f-a4d4-74e2d7c21133 service nova] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:08:41.718405 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-fe4a80d0-1580-4a3f-a4d4-74e2d7c21133 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:08:41.825218 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a80d0-1580-4a3f-a4d4-74e2d7c21133 service nova] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:08:41.832303 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98bfe5-e510-4456-98dd-17bbf5622584 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887a619ef5b5417db98ff15a9f712724', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:08:41.833574 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98bfe5-e510-4456-98dd-17bbf5622584 service placement] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:08:41.833850 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98bfe5-e510-4456-98dd-17bbf5622584 service placement] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:08:41.834775 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98bfe5-e510-4456-98dd-17bbf5622584 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:08:41.837043 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-8d98bfe5-e510-4456-98dd-17bbf5622584 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:08:41.851273 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d98bfe5-e510-4456-98dd-17bbf5622584 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:08:41.852925 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d98bfe5-e510-4456-98dd-17bbf5622584 service placement] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:08:41.867663 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1364/2745] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:08:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 15:08:44.141056 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7830197-9ec0-4a0b-bec4-c0a1c10aeed9 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:08:44.162463 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7830197-9ec0-4a0b-bec4-c0a1c10aeed9 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:08:44.169952 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f967fee-e1bd-4d56-a506-7c378b50455f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:08:44.171367 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f967fee-e1bd-4d56-a506-7c378b50455f service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:08:44.171826 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f967fee-e1bd-4d56-a506-7c378b50455f service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:08:44.172275 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f967fee-e1bd-4d56-a506-7c378b50455f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:08:44.176011 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-3f967fee-e1bd-4d56-a506-7c378b50455f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:08:44.191631 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f967fee-e1bd-4d56-a506-7c378b50455f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:08:44.193596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f967fee-e1bd-4d56-a506-7c378b50455f service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:08:44.203214 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1382/2746] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:08:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 15:09:28.812525 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98bfe5-e510-4456-98dd-17bbf5622584 service placement] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:09:28.823147 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98bfe5-e510-4456-98dd-17bbf5622584 service placement] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:09:28.826753 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a30ce-3e16-433d-81a8-90bcf7272a8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:09:28.827399 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a30ce-3e16-433d-81a8-90bcf7272a8d service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:09:28.827676 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a30ce-3e16-433d-81a8-90bcf7272a8d service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:09:28.827944 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a30ce-3e16-433d-81a8-90bcf7272a8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:09:28.829856 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-596a30ce-3e16-433d-81a8-90bcf7272a8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:09:28.895916 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596a30ce-3e16-433d-81a8-90bcf7272a8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:09:28.897390 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596a30ce-3e16-433d-81a8-90bcf7272a8d service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:09:28.907663 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1365/2747] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:09:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 15:09:30.220017 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f967fee-e1bd-4d56-a506-7c378b50455f service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:09:30.237294 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f967fee-e1bd-4d56-a506-7c378b50455f service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:09:30.243822 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192157b-4e82-417c-be0b-76838628f34b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:09:30.244734 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192157b-4e82-417c-be0b-76838628f34b service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:09:30.245157 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192157b-4e82-417c-be0b-76838628f34b service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:09:30.245575 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192157b-4e82-417c-be0b-76838628f34b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:09:30.247879 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-6192157b-4e82-417c-be0b-76838628f34b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:09:30.263185 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6192157b-4e82-417c-be0b-76838628f34b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:09:30.265727 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6192157b-4e82-417c-be0b-76838628f34b service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:09:30.283326 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1383/2748] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:09:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 15:09:35.402318 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a30ce-3e16-433d-81a8-90bcf7272a8d service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:09:35.412323 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a30ce-3e16-433d-81a8-90bcf7272a8d service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:09:35.415934 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e8b024-f123-4fef-ab68-0ca2a8cef4d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:09:35.416569 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e8b024-f123-4fef-ab68-0ca2a8cef4d1 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:09:35.416874 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e8b024-f123-4fef-ab68-0ca2a8cef4d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:09:35.417168 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e8b024-f123-4fef-ab68-0ca2a8cef4d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:09:35.418891 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-86e8b024-f123-4fef-ab68-0ca2a8cef4d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:09:35.432246 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e8b024-f123-4fef-ab68-0ca2a8cef4d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:09:35.433743 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e8b024-f123-4fef-ab68-0ca2a8cef4d1 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:09:35.444676 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1366/2749] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:09:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:09:42.754489 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192157b-4e82-417c-be0b-76838628f34b service nova] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:09:42.756032 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-6192157b-4e82-417c-be0b-76838628f34b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:09:42.779317 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192157b-4e82-417c-be0b-76838628f34b service nova] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:09:42.785203 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ed4481-ae3a-4aa7-98eb-eaa051866013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:09:42.785786 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed4481-ae3a-4aa7-98eb-eaa051866013 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:09:42.786153 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed4481-ae3a-4aa7-98eb-eaa051866013 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:09:42.786630 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed4481-ae3a-4aa7-98eb-eaa051866013 admin admin] PATH_INFO: `/v3/users/f9f347a369724fcea691938a0391e05e` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:09:42.787954 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ed4481-ae3a-4aa7-98eb-eaa051866013 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9f347a369724fcea691938a0391e05e)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:09:42.790111 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ed4481-ae3a-4aa7-98eb-eaa051866013 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:09:42.839824 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-89ed4481-ae3a-4aa7-98eb-eaa051866013 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9f347a369724fcea691938a0391e05e'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:09:42.849660 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-89ed4481-ae3a-4aa7-98eb-eaa051866013 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9f347a369724fcea691938a0391e05e'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:09:42.857502 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-89ed4481-ae3a-4aa7-98eb-eaa051866013 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9f347a369724fcea691938a0391e05e'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:09:42.858227 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-89ed4481-ae3a-4aa7-98eb-eaa051866013 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9f347a369724fcea691938a0391e05e'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:09:42.858850 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-89ed4481-ae3a-4aa7-98eb-eaa051866013 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9f347a369724fcea691938a0391e05e'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:09:42.879246 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1384/2750] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:09:42 2022] DELETE /identity/v3/users/f9f347a369724fcea691938a0391e05e => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:09:43.095946 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e8b024-f123-4fef-ab68-0ca2a8cef4d1 service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:09:43.105856 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e8b024-f123-4fef-ab68-0ca2a8cef4d1 service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:09:43.170137 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d13dd-7217-46f8-9289-6c5aad351182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:09:43.171008 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d13dd-7217-46f8-9289-6c5aad351182 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:09:43.171293 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d13dd-7217-46f8-9289-6c5aad351182 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:09:43.171573 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d13dd-7217-46f8-9289-6c5aad351182 admin admin] PATH_INFO: `/v3/projects/32f5765bb183428c8a04aed1c66a2068` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:09:43.173601 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0d13dd-7217-46f8-9289-6c5aad351182 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32f5765bb183428c8a04aed1c66a2068)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:09:43.175528 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0d13dd-7217-46f8-9289-6c5aad351182 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:09:43.224269 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-aa0d13dd-7217-46f8-9289-6c5aad351182 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32f5765bb183428c8a04aed1c66a2068', 'request_id': 'req-aa0d13dd-7217-46f8-9289-6c5aad351182'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:09:43.229490 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-aa0d13dd-7217-46f8-9289-6c5aad351182 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32f5765bb183428c8a04aed1c66a2068', 'request_id': 'req-aa0d13dd-7217-46f8-9289-6c5aad351182'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:09:43.238700 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-aa0d13dd-7217-46f8-9289-6c5aad351182 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32f5765bb183428c8a04aed1c66a2068', 'request_id': 'req-aa0d13dd-7217-46f8-9289-6c5aad351182'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:09:43.250865 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-aa0d13dd-7217-46f8-9289-6c5aad351182 admin admin] The token cache is being invalidate because project 32f5765bb183428c8a04aed1c66a2068 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:09:43.251156 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-aa0d13dd-7217-46f8-9289-6c5aad351182 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:09:43.251926 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-aa0d13dd-7217-46f8-9289-6c5aad351182 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:09:43.253659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1367/2751] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:09:43 2022] DELETE /identity/v3/projects/32f5765bb183428c8a04aed1c66a2068 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:09:43.258106 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ed4481-ae3a-4aa7-98eb-eaa051866013 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:09:43.264203 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-89ed4481-ae3a-4aa7-98eb-eaa051866013 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:09:43.278680 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ed4481-ae3a-4aa7-98eb-eaa051866013 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:09:43.340560 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779585ff-4faf-4022-b0f2-45e50240865b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:09:43.341526 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779585ff-4faf-4022-b0f2-45e50240865b admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:09:43.341811 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779585ff-4faf-4022-b0f2-45e50240865b admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:09:43.342167 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779585ff-4faf-4022-b0f2-45e50240865b admin admin] PATH_INFO: `/v3/users/9af2da9ba76e48b497c16cb152eaa8a5` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:09:43.343461 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779585ff-4faf-4022-b0f2-45e50240865b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9af2da9ba76e48b497c16cb152eaa8a5)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:09:43.346089 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779585ff-4faf-4022-b0f2-45e50240865b admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:09:43.395475 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-779585ff-4faf-4022-b0f2-45e50240865b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9af2da9ba76e48b497c16cb152eaa8a5'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:09:43.402991 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-779585ff-4faf-4022-b0f2-45e50240865b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9af2da9ba76e48b497c16cb152eaa8a5'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:09:43.410864 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-779585ff-4faf-4022-b0f2-45e50240865b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9af2da9ba76e48b497c16cb152eaa8a5'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:09:43.411820 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-779585ff-4faf-4022-b0f2-45e50240865b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9af2da9ba76e48b497c16cb152eaa8a5'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:09:43.412247 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-779585ff-4faf-4022-b0f2-45e50240865b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9af2da9ba76e48b497c16cb152eaa8a5'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:09:43.432418 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1385/2752] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:09:43 2022] DELETE /identity/v3/users/9af2da9ba76e48b497c16cb152eaa8a5 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:09:43.642515 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d13dd-7217-46f8-9289-6c5aad351182 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:09:43.644845 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-aa0d13dd-7217-46f8-9289-6c5aad351182 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:09:43.662660 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d13dd-7217-46f8-9289-6c5aad351182 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:09:43.726503 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dc48e5-5ce5-4a7d-82f8-62e4c05710ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:09:43.727378 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc48e5-5ce5-4a7d-82f8-62e4c05710ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:09:43.727666 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc48e5-5ce5-4a7d-82f8-62e4c05710ca admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:09:43.727948 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc48e5-5ce5-4a7d-82f8-62e4c05710ca admin admin] PATH_INFO: `/v3/projects/a62c7d504fd44db9a30bea8d201735c6` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:09:43.729182 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dc48e5-5ce5-4a7d-82f8-62e4c05710ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=a62c7d504fd44db9a30bea8d201735c6)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:09:43.731088 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dc48e5-5ce5-4a7d-82f8-62e4c05710ca admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:09:43.784410 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-47dc48e5-5ce5-4a7d-82f8-62e4c05710ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a62c7d504fd44db9a30bea8d201735c6', 'request_id': 'req-47dc48e5-5ce5-4a7d-82f8-62e4c05710ca'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:09:43.790754 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-47dc48e5-5ce5-4a7d-82f8-62e4c05710ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a62c7d504fd44db9a30bea8d201735c6', 'request_id': 'req-47dc48e5-5ce5-4a7d-82f8-62e4c05710ca'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:09:43.801274 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-47dc48e5-5ce5-4a7d-82f8-62e4c05710ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a62c7d504fd44db9a30bea8d201735c6', 'request_id': 'req-47dc48e5-5ce5-4a7d-82f8-62e4c05710ca'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:09:43.814575 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-47dc48e5-5ce5-4a7d-82f8-62e4c05710ca admin admin] The token cache is being invalidate because project a62c7d504fd44db9a30bea8d201735c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:09:43.814869 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-47dc48e5-5ce5-4a7d-82f8-62e4c05710ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:09:43.815581 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-47dc48e5-5ce5-4a7d-82f8-62e4c05710ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:09:43.817252 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1368/2753] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:09:43 2022] DELETE /identity/v3/projects/a62c7d504fd44db9a30bea8d201735c6 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:09:50.021511 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab54927-d2c6-41ef-82b3-f37d5465557b None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:09:50.022161 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab54927-d2c6-41ef-82b3-f37d5465557b None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:09:50.022161 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab54927-d2c6-41ef-82b3-f37d5465557b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:09:50.042459 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-7ab54927-d2c6-41ef-82b3-f37d5465557b None None] MFA Rules not processed for user `90accf4926bf452d8ff6e79a1c784911`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:09:50.043700 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-7ab54927-d2c6-41ef-82b3-f37d5465557b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:09:50.108167 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1386/2754] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:09:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:09:53.774456 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dc48e5-5ce5-4a7d-82f8-62e4c05710ca admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:09:53.779905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-47dc48e5-5ce5-4a7d-82f8-62e4c05710ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:09:53.940728 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dc48e5-5ce5-4a7d-82f8-62e4c05710ca admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:09:53.951350 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc50a4b-300d-49a7-a3d0-6a89ef5679d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:09:53.952783 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc50a4b-300d-49a7-a3d0-6a89ef5679d9 service nova] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:09:53.954236 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc50a4b-300d-49a7-a3d0-6a89ef5679d9 service nova] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:09:53.954760 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc50a4b-300d-49a7-a3d0-6a89ef5679d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:09:53.960018 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-cfc50a4b-300d-49a7-a3d0-6a89ef5679d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:09:53.981408 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc50a4b-300d-49a7-a3d0-6a89ef5679d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:09:53.983752 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc50a4b-300d-49a7-a3d0-6a89ef5679d9 service nova] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:09:53.997124 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1369/2755] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:09:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:09:59.070668 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab54927-d2c6-41ef-82b3-f37d5465557b None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:09:59.072158 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-7ab54927-d2c6-41ef-82b3-f37d5465557b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:09:59.096902 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab54927-d2c6-41ef-82b3-f37d5465557b None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:09:59.103800 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8342fcc-d919-426a-b8d2-5373a557ae7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc8cf09642ec4e10a80badcf8b585ef8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:09:59.104472 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8342fcc-d919-426a-b8d2-5373a557ae7e service nova] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:09:59.104817 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8342fcc-d919-426a-b8d2-5373a557ae7e service nova] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:09:59.105097 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8342fcc-d919-426a-b8d2-5373a557ae7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:09:59.107015 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e8342fcc-d919-426a-b8d2-5373a557ae7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:09:59.120986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8342fcc-d919-426a-b8d2-5373a557ae7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:09:59.122623 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8342fcc-d919-426a-b8d2-5373a557ae7e service nova] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:09:59.139790 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1387/2756] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:09:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 15:10:05.635890 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc50a4b-300d-49a7-a3d0-6a89ef5679d9 service nova] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:10:05.637482 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-cfc50a4b-300d-49a7-a3d0-6a89ef5679d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:10:05.706064 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc50a4b-300d-49a7-a3d0-6a89ef5679d9 service nova] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:10:05.709788 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b184479c-12a1-41d6-bfad-e18ae3f5faeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:10:05.710450 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184479c-12a1-41d6-bfad-e18ae3f5faeb service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:10:05.710728 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184479c-12a1-41d6-bfad-e18ae3f5faeb service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:10:05.711002 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184479c-12a1-41d6-bfad-e18ae3f5faeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:10:05.712942 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b184479c-12a1-41d6-bfad-e18ae3f5faeb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:10:05.809614 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b184479c-12a1-41d6-bfad-e18ae3f5faeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:10:05.812932 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b184479c-12a1-41d6-bfad-e18ae3f5faeb service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:10:05.848940 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1370/2757] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:10:05 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:10:19.974211 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8342fcc-d919-426a-b8d2-5373a557ae7e service nova] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:10:19.977140 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-e8342fcc-d919-426a-b8d2-5373a557ae7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:10:20.048327 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8342fcc-d919-426a-b8d2-5373a557ae7e service nova] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:10:20.052321 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e22530-5c4b-4646-be61-433ae73b2c54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:10:20.053119 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e22530-5c4b-4646-be61-433ae73b2c54 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:10:20.053441 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e22530-5c4b-4646-be61-433ae73b2c54 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:10:20.053769 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e22530-5c4b-4646-be61-433ae73b2c54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:10:20.055833 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-b7e22530-5c4b-4646-be61-433ae73b2c54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:10:20.208783 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e22530-5c4b-4646-be61-433ae73b2c54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:10:20.210229 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e22530-5c4b-4646-be61-433ae73b2c54 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:10:20.232540 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1388/2758] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:10:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 15:10:28.153768 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b184479c-12a1-41d6-bfad-e18ae3f5faeb service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:10:28.158467 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b184479c-12a1-41d6-bfad-e18ae3f5faeb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:10:28.210041 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b184479c-12a1-41d6-bfad-e18ae3f5faeb service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:10:28.215545 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370efb52-c7c4-495c-adb3-215001b79220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:10:28.217225 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370efb52-c7c4-495c-adb3-215001b79220 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:10:28.217676 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370efb52-c7c4-495c-adb3-215001b79220 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:10:28.218646 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370efb52-c7c4-495c-adb3-215001b79220 admin admin] PATH_INFO: `/v3/users/4b3af1d82c33446e998f07bce54ef941` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:10:28.220792 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370efb52-c7c4-495c-adb3-215001b79220 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b3af1d82c33446e998f07bce54ef941)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:10:28.223240 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370efb52-c7c4-495c-adb3-215001b79220 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:10:28.277171 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-370efb52-c7c4-495c-adb3-215001b79220 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b3af1d82c33446e998f07bce54ef941'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:10:28.278647 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-370efb52-c7c4-495c-adb3-215001b79220 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b3af1d82c33446e998f07bce54ef941'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:10:28.289363 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-370efb52-c7c4-495c-adb3-215001b79220 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b3af1d82c33446e998f07bce54ef941'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:10:28.303164 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-370efb52-c7c4-495c-adb3-215001b79220 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b3af1d82c33446e998f07bce54ef941'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:10:28.304317 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-370efb52-c7c4-495c-adb3-215001b79220 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b3af1d82c33446e998f07bce54ef941'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:10:28.337025 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1371/2759] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:10:28 2022] DELETE /identity/v3/users/4b3af1d82c33446e998f07bce54ef941 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:10:28.548834 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e22530-5c4b-4646-be61-433ae73b2c54 service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:10:28.560774 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e22530-5c4b-4646-be61-433ae73b2c54 service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:10:28.637800 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9642945-f5a3-4d59-9405-503741491786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:10:28.638598 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9642945-f5a3-4d59-9405-503741491786 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:10:28.638904 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9642945-f5a3-4d59-9405-503741491786 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:10:28.639162 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9642945-f5a3-4d59-9405-503741491786 admin admin] PATH_INFO: `/v3/projects/bb5f41fdaad246b7bc30c8f8021129a3` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:10:28.640499 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9642945-f5a3-4d59-9405-503741491786 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb5f41fdaad246b7bc30c8f8021129a3)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:10:28.642552 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9642945-f5a3-4d59-9405-503741491786 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:10:28.707724 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c9642945-f5a3-4d59-9405-503741491786 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb5f41fdaad246b7bc30c8f8021129a3', 'request_id': 'req-c9642945-f5a3-4d59-9405-503741491786'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:10:28.715707 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c9642945-f5a3-4d59-9405-503741491786 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb5f41fdaad246b7bc30c8f8021129a3', 'request_id': 'req-c9642945-f5a3-4d59-9405-503741491786'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:10:28.725907 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c9642945-f5a3-4d59-9405-503741491786 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb5f41fdaad246b7bc30c8f8021129a3', 'request_id': 'req-c9642945-f5a3-4d59-9405-503741491786'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:10:28.740041 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c9642945-f5a3-4d59-9405-503741491786 admin admin] The token cache is being invalidate because project bb5f41fdaad246b7bc30c8f8021129a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:10:28.740315 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c9642945-f5a3-4d59-9405-503741491786 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:10:28.741014 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-c9642945-f5a3-4d59-9405-503741491786 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:10:28.742621 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1389/2760] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:10:28 2022] DELETE /identity/v3/projects/bb5f41fdaad246b7bc30c8f8021129a3 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:10:28.746781 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370efb52-c7c4-495c-adb3-215001b79220 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:10:28.748281 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-370efb52-c7c4-495c-adb3-215001b79220 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:10:28.763116 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370efb52-c7c4-495c-adb3-215001b79220 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:10:28.822960 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01645036-d76c-43a5-9299-22fbcd15b23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:10:28.823836 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01645036-d76c-43a5-9299-22fbcd15b23b admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:10:28.824114 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01645036-d76c-43a5-9299-22fbcd15b23b admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:10:28.824382 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01645036-d76c-43a5-9299-22fbcd15b23b admin admin] PATH_INFO: `/v3/users/8959632f4e8e4f61bf569d3fa3c32719` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:10:28.840437 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01645036-d76c-43a5-9299-22fbcd15b23b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8959632f4e8e4f61bf569d3fa3c32719)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:10:28.842459 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01645036-d76c-43a5-9299-22fbcd15b23b admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:10:28.897827 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-01645036-d76c-43a5-9299-22fbcd15b23b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8959632f4e8e4f61bf569d3fa3c32719'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:10:28.898466 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-01645036-d76c-43a5-9299-22fbcd15b23b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8959632f4e8e4f61bf569d3fa3c32719'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:10:28.907449 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-01645036-d76c-43a5-9299-22fbcd15b23b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8959632f4e8e4f61bf569d3fa3c32719'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:10:28.915280 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-01645036-d76c-43a5-9299-22fbcd15b23b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8959632f4e8e4f61bf569d3fa3c32719'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:10:28.916031 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-01645036-d76c-43a5-9299-22fbcd15b23b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8959632f4e8e4f61bf569d3fa3c32719'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:10:28.934403 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1372/2761] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:10:28 2022] DELETE /identity/v3/users/8959632f4e8e4f61bf569d3fa3c32719 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 15:10:29.130716 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9642945-f5a3-4d59-9405-503741491786 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:10:29.132382 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-c9642945-f5a3-4d59-9405-503741491786 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:10:29.150761 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9642945-f5a3-4d59-9405-503741491786 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:10:29.217519 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07ab3d1-22bd-430f-9578-ae39d58c4808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:10:29.218985 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ab3d1-22bd-430f-9578-ae39d58c4808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:10:29.219259 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ab3d1-22bd-430f-9578-ae39d58c4808 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:10:29.219529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ab3d1-22bd-430f-9578-ae39d58c4808 admin admin] PATH_INFO: `/v3/projects/9835badd92d54c2b9c31ed869e7ba010` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:10:29.231389 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07ab3d1-22bd-430f-9578-ae39d58c4808 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9835badd92d54c2b9c31ed869e7ba010)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:10:29.233385 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07ab3d1-22bd-430f-9578-ae39d58c4808 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:10:29.286235 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f07ab3d1-22bd-430f-9578-ae39d58c4808 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9835badd92d54c2b9c31ed869e7ba010', 'request_id': 'req-f07ab3d1-22bd-430f-9578-ae39d58c4808'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:10:29.292552 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f07ab3d1-22bd-430f-9578-ae39d58c4808 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9835badd92d54c2b9c31ed869e7ba010', 'request_id': 'req-f07ab3d1-22bd-430f-9578-ae39d58c4808'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:10:29.301077 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f07ab3d1-22bd-430f-9578-ae39d58c4808 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9835badd92d54c2b9c31ed869e7ba010', 'request_id': 'req-f07ab3d1-22bd-430f-9578-ae39d58c4808'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:10:29.314860 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f07ab3d1-22bd-430f-9578-ae39d58c4808 admin admin] The token cache is being invalidate because project 9835badd92d54c2b9c31ed869e7ba010 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:10:29.315147 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f07ab3d1-22bd-430f-9578-ae39d58c4808 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:10:29.315828 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-f07ab3d1-22bd-430f-9578-ae39d58c4808 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:10:29.317475 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1390/2762] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:10:29 2022] DELETE /identity/v3/projects/9835badd92d54c2b9c31ed869e7ba010 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:11:00.108213 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a3d2c-3dc3-4176-9e3f-d28d76468ef8 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:11:00.109012 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a3d2c-3dc3-4176-9e3f-d28d76468ef8 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:11:00.109012 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a3d2c-3dc3-4176-9e3f-d28d76468ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:11:00.126361 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-b75a3d2c-3dc3-4176-9e3f-d28d76468ef8 None None] MFA Rules not processed for user `90accf4926bf452d8ff6e79a1c784911`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:11:00.127570 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b75a3d2c-3dc3-4176-9e3f-d28d76468ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:11:00.193975 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1373/2763] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:11:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:11:00.530298 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07ab3d1-22bd-430f-9578-ae39d58c4808 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:11:00.532914 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f07ab3d1-22bd-430f-9578-ae39d58c4808 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:11:00.630595 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07ab3d1-22bd-430f-9578-ae39d58c4808 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:11:00.634663 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bccd157-7016-4dab-b515-bea87620e780 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:11:00.635348 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bccd157-7016-4dab-b515-bea87620e780 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:11:00.635638 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bccd157-7016-4dab-b515-bea87620e780 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:11:00.635913 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bccd157-7016-4dab-b515-bea87620e780 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:11:00.637958 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-4bccd157-7016-4dab-b515-bea87620e780 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:11:00.648992 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bccd157-7016-4dab-b515-bea87620e780 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:11:00.650491 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bccd157-7016-4dab-b515-bea87620e780 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:11:00.660489 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1391/2764] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:11:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:11:33.381186 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a3d2c-3dc3-4176-9e3f-d28d76468ef8 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:11:33.382750 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-b75a3d2c-3dc3-4176-9e3f-d28d76468ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:11:33.491084 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a3d2c-3dc3-4176-9e3f-d28d76468ef8 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:11:33.496582 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c6cad-bd2c-4dcc-88b5-a863de2da6a7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887a619ef5b5417db98ff15a9f712724', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:11:33.497335 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c6cad-bd2c-4dcc-88b5-a863de2da6a7 service placement] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:11:33.498198 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c6cad-bd2c-4dcc-88b5-a863de2da6a7 service placement] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:11:33.498671 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c6cad-bd2c-4dcc-88b5-a863de2da6a7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:11:33.500828 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ad9c6cad-bd2c-4dcc-88b5-a863de2da6a7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:11:33.514353 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9c6cad-bd2c-4dcc-88b5-a863de2da6a7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:11:33.516009 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9c6cad-bd2c-4dcc-88b5-a863de2da6a7 service placement] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:11:33.527243 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1374/2765] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:11:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:12:03.572167 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3353e-4f26-4f3a-939a-5d0e3963cb34 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:12:03.572821 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3353e-4f26-4f3a-939a-5d0e3963cb34 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:12:03.572821 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3353e-4f26-4f3a-939a-5d0e3963cb34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:12:03.589791 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-25d3353e-4f26-4f3a-939a-5d0e3963cb34 None None] MFA Rules not processed for user `90accf4926bf452d8ff6e79a1c784911`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:12:03.590991 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-25d3353e-4f26-4f3a-939a-5d0e3963cb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:12:03.597924 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1392/2766] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:12:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:12:03.901564 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c6cad-bd2c-4dcc-88b5-a863de2da6a7 service placement] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:12:03.903073 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ad9c6cad-bd2c-4dcc-88b5-a863de2da6a7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:12:03.914702 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c6cad-bd2c-4dcc-88b5-a863de2da6a7 service placement] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:12:03.918265 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cafb38e-d9b7-4f97-af1e-92bdb40cdead service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:12:03.919394 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cafb38e-d9b7-4f97-af1e-92bdb40cdead service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:12:03.919672 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cafb38e-d9b7-4f97-af1e-92bdb40cdead service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:12:03.919946 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cafb38e-d9b7-4f97-af1e-92bdb40cdead service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:12:03.921650 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-5cafb38e-d9b7-4f97-af1e-92bdb40cdead service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:12:03.932017 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cafb38e-d9b7-4f97-af1e-92bdb40cdead service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:12:03.934610 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cafb38e-d9b7-4f97-af1e-92bdb40cdead service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:12:03.946962 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1375/2767] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:12:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:13:12.783457 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3353e-4f26-4f3a-939a-5d0e3963cb34 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:12.785048 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-25d3353e-4f26-4f3a-939a-5d0e3963cb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:13:12.857399 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3353e-4f26-4f3a-939a-5d0e3963cb34 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:12.861828 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b07cbff-7824-400d-a1e4-4a2c924e3dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:12.862787 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07cbff-7824-400d-a1e4-4a2c924e3dc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:12.863072 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07cbff-7824-400d-a1e4-4a2c924e3dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:12.863458 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07cbff-7824-400d-a1e4-4a2c924e3dc7 admin admin] PATH_INFO: `/v3/users/90accf4926bf452d8ff6e79a1c784911` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:12.864713 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b07cbff-7824-400d-a1e4-4a2c924e3dc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90accf4926bf452d8ff6e79a1c784911)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:12.866610 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b07cbff-7824-400d-a1e4-4a2c924e3dc7 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:12.912399 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-7b07cbff-7824-400d-a1e4-4a2c924e3dc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90accf4926bf452d8ff6e79a1c784911'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:12.919771 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-7b07cbff-7824-400d-a1e4-4a2c924e3dc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90accf4926bf452d8ff6e79a1c784911'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:12.926850 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-7b07cbff-7824-400d-a1e4-4a2c924e3dc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90accf4926bf452d8ff6e79a1c784911'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:12.927499 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-7b07cbff-7824-400d-a1e4-4a2c924e3dc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90accf4926bf452d8ff6e79a1c784911'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:12.928099 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-7b07cbff-7824-400d-a1e4-4a2c924e3dc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90accf4926bf452d8ff6e79a1c784911'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:12.945288 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1393/2768] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:13:12 2022] DELETE /identity/v3/users/90accf4926bf452d8ff6e79a1c784911 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:13:13.118780 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cafb38e-d9b7-4f97-af1e-92bdb40cdead service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:13.133275 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cafb38e-d9b7-4f97-af1e-92bdb40cdead service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:13.192750 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071cae90-5560-4f0c-89c4-6493b1f9c4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:13.193614 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071cae90-5560-4f0c-89c4-6493b1f9c4db admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:13.193903 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071cae90-5560-4f0c-89c4-6493b1f9c4db admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:13.194179 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071cae90-5560-4f0c-89c4-6493b1f9c4db admin admin] PATH_INFO: `/v3/projects/6f2937968fc04cfaa3dab933e994a975` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:13.195358 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071cae90-5560-4f0c-89c4-6493b1f9c4db admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f2937968fc04cfaa3dab933e994a975)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:13.197289 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071cae90-5560-4f0c-89c4-6493b1f9c4db admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:13.241935 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-071cae90-5560-4f0c-89c4-6493b1f9c4db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f2937968fc04cfaa3dab933e994a975', 'request_id': 'req-071cae90-5560-4f0c-89c4-6493b1f9c4db'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:13.247722 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-071cae90-5560-4f0c-89c4-6493b1f9c4db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f2937968fc04cfaa3dab933e994a975', 'request_id': 'req-071cae90-5560-4f0c-89c4-6493b1f9c4db'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:13.256283 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-071cae90-5560-4f0c-89c4-6493b1f9c4db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f2937968fc04cfaa3dab933e994a975', 'request_id': 'req-071cae90-5560-4f0c-89c4-6493b1f9c4db'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:13.267743 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-071cae90-5560-4f0c-89c4-6493b1f9c4db admin admin] The token cache is being invalidate because project 6f2937968fc04cfaa3dab933e994a975 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:13:13.268029 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-071cae90-5560-4f0c-89c4-6493b1f9c4db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:13.268713 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-071cae90-5560-4f0c-89c4-6493b1f9c4db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:13.270354 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1376/2769] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:13:13 2022] DELETE /identity/v3/projects/6f2937968fc04cfaa3dab933e994a975 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:13:13.275444 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd00108-2e4c-47ac-a300-f2203afee81f None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:13.275741 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd00108-2e4c-47ac-a300-f2203afee81f None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:13.276018 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd00108-2e4c-47ac-a300-f2203afee81f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:13.333540 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-1dd00108-2e4c-47ac-a300-f2203afee81f None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:13:13.334718 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-1dd00108-2e4c-47ac-a300-f2203afee81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:13:13.396874 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1394/2770] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:13:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:13:13.405576 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf9df8-dd37-49a4-8c66-85e0e376a929 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:13.405869 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf9df8-dd37-49a4-8c66-85e0e376a929 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:13.406147 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf9df8-dd37-49a4-8c66-85e0e376a929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:13.425727 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-72cf9df8-dd37-49a4-8c66-85e0e376a929 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:13:13.426931 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-72cf9df8-dd37-49a4-8c66-85e0e376a929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:13:13.435351 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1377/2771] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:13:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:13:13.439671 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd00108-2e4c-47ac-a300-f2203afee81f None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:13.449374 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd00108-2e4c-47ac-a300-f2203afee81f None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:13.454312 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b20dc6-9b0e-4528-938d-d556371dedfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:13.454975 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b20dc6-9b0e-4528-938d-d556371dedfa admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:13.455260 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b20dc6-9b0e-4528-938d-d556371dedfa admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:13.455539 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b20dc6-9b0e-4528-938d-d556371dedfa admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:13.456262 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b20dc6-9b0e-4528-938d-d556371dedfa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 15:13:13.456567 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b20dc6-9b0e-4528-938d-d556371dedfa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:13.458204 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b20dc6-9b0e-4528-938d-d556371dedfa admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:13.470346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1395/2772] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 15:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 15:13:13.474711 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf9df8-dd37-49a4-8c66-85e0e376a929 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:13.484532 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf9df8-dd37-49a4-8c66-85e0e376a929 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:13.489193 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56e862-d80f-4e83-a7c3-94a223a08c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:13.489841 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56e862-d80f-4e83-a7c3-94a223a08c54 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:13.490124 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56e862-d80f-4e83-a7c3-94a223a08c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:13.490400 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56e862-d80f-4e83-a7c3-94a223a08c54 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:13.491104 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a56e862-d80f-4e83-a7c3-94a223a08c54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:13.493065 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a56e862-d80f-4e83-a7c3-94a223a08c54 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:13.518204 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1378/2773] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:13:13 2022] POST /identity/v3/projects => generated 381 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:13:13.522400 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b20dc6-9b0e-4528-938d-d556371dedfa admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:13.532765 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b20dc6-9b0e-4528-938d-d556371dedfa admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:13.588928 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07e9130-99cc-4e4c-a6ec-36c615edbf2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:13.589510 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07e9130-99cc-4e4c-a6ec-36c615edbf2e admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:13.589791 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07e9130-99cc-4e4c-a6ec-36c615edbf2e admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:13.590065 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07e9130-99cc-4e4c-a6ec-36c615edbf2e admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:13.590848 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07e9130-99cc-4e4c-a6ec-36c615edbf2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:13.592925 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07e9130-99cc-4e4c-a6ec-36c615edbf2e admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:13.612729 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1396/2774] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:13:13 2022] POST /identity/v3/users => generated 291 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 15:13:13.616807 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56e862-d80f-4e83-a7c3-94a223a08c54 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:13.628321 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56e862-d80f-4e83-a7c3-94a223a08c54 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:13.632971 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3514dab-cad1-459a-bee0-0658c9009f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:13.633624 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3514dab-cad1-459a-bee0-0658c9009f13 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:13.633902 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3514dab-cad1-459a-bee0-0658c9009f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:13.634173 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3514dab-cad1-459a-bee0-0658c9009f13 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:13.634883 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3514dab-cad1-459a-bee0-0658c9009f13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:13.636454 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3514dab-cad1-459a-bee0-0658c9009f13 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:13.647640 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1379/2775] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:13:13.651635 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07e9130-99cc-4e4c-a6ec-36c615edbf2e admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:13.662588 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07e9130-99cc-4e4c-a6ec-36c615edbf2e admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:13.667215 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24037d4f-270e-4eab-8dec-f3455b30e9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:13.667873 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24037d4f-270e-4eab-8dec-f3455b30e9bb admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:13.668158 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24037d4f-270e-4eab-8dec-f3455b30e9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:13.668471 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24037d4f-270e-4eab-8dec-f3455b30e9bb admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:13.669960 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24037d4f-270e-4eab-8dec-f3455b30e9bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:13.671545 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24037d4f-270e-4eab-8dec-f3455b30e9bb admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:13.681507 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1397/2776] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:13:13.686416 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3514dab-cad1-459a-bee0-0658c9009f13 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:13.696187 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3514dab-cad1-459a-bee0-0658c9009f13 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:13.701339 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6d965-70ca-450a-976c-2c287690b71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:13.702258 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6d965-70ca-450a-976c-2c287690b71e admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:13.702536 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6d965-70ca-450a-976c-2c287690b71e admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:13.702824 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6d965-70ca-450a-976c-2c287690b71e admin admin] PATH_INFO: `/v3/projects/7844651fb3904bca917af1b37735afa0/users/f32dac66fccf413f8255507011eee5ce/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:13.719664 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db6d965-70ca-450a-976c-2c287690b71e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7844651fb3904bca917af1b37735afa0, user_id=f32dac66fccf413f8255507011eee5ce, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:13.722359 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db6d965-70ca-450a-976c-2c287690b71e admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:13.735835 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1380/2777] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:13:13 2022] PUT /identity/v3/projects/7844651fb3904bca917af1b37735afa0/users/f32dac66fccf413f8255507011eee5ce/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:13:13.746038 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71282a-cb8b-4827-a612-a28c82987725 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:13.746332 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71282a-cb8b-4827-a612-a28c82987725 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:13.746612 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71282a-cb8b-4827-a612-a28c82987725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:13.773396 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-ee71282a-cb8b-4827-a612-a28c82987725 None None] MFA Rules not processed for user `f32dac66fccf413f8255507011eee5ce`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:13:13.774718 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-ee71282a-cb8b-4827-a612-a28c82987725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:13:13.841670 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1398/2778] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:13:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:13:13.846413 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6d965-70ca-450a-976c-2c287690b71e admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:13.908178 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6d965-70ca-450a-976c-2c287690b71e admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:13.912829 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede1a2ec-24bb-4e53-8e94-f0b31d0b36c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:13.913487 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede1a2ec-24bb-4e53-8e94-f0b31d0b36c5 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:13.913773 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede1a2ec-24bb-4e53-8e94-f0b31d0b36c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:13.914050 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede1a2ec-24bb-4e53-8e94-f0b31d0b36c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:13.914738 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede1a2ec-24bb-4e53-8e94-f0b31d0b36c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:13.916652 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede1a2ec-24bb-4e53-8e94-f0b31d0b36c5 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:13.942052 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1381/2779] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:13:13 2022] POST /identity/v3/projects => generated 381 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:13:13.946069 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee71282a-cb8b-4827-a612-a28c82987725 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:13.957273 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee71282a-cb8b-4827-a612-a28c82987725 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:14.015509 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f433920a-0d8a-4d89-92bf-da4fbedd33ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:14.016194 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433920a-0d8a-4d89-92bf-da4fbedd33ff admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:14.016478 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433920a-0d8a-4d89-92bf-da4fbedd33ff admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:14.016790 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433920a-0d8a-4d89-92bf-da4fbedd33ff admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:14.017487 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f433920a-0d8a-4d89-92bf-da4fbedd33ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:14.019403 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f433920a-0d8a-4d89-92bf-da4fbedd33ff admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:14.041052 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1399/2780] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:13:13 2022] POST /identity/v3/users => generated 297 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 15:13:14.045041 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede1a2ec-24bb-4e53-8e94-f0b31d0b36c5 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:14.055346 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede1a2ec-24bb-4e53-8e94-f0b31d0b36c5 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:14.060020 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113313c2-87bd-4997-97c2-e265f4a26a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:14.060709 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113313c2-87bd-4997-97c2-e265f4a26a66 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:14.060997 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113313c2-87bd-4997-97c2-e265f4a26a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:14.061272 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113313c2-87bd-4997-97c2-e265f4a26a66 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:14.061993 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113313c2-87bd-4997-97c2-e265f4a26a66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:14.063643 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113313c2-87bd-4997-97c2-e265f4a26a66 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:14.075813 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1382/2781] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:13:14.080439 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f433920a-0d8a-4d89-92bf-da4fbedd33ff admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:14.091740 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f433920a-0d8a-4d89-92bf-da4fbedd33ff admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:14.096277 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81082650-c8ea-4a43-b9a1-9b2cc0596d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:14.097188 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81082650-c8ea-4a43-b9a1-9b2cc0596d7b admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:14.097468 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81082650-c8ea-4a43-b9a1-9b2cc0596d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:14.097743 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81082650-c8ea-4a43-b9a1-9b2cc0596d7b admin admin] PATH_INFO: `/v3/projects/06ab191e97214294853e317575a92daa/users/caf0699a2ddf449194e56195e6e5e5c6/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:14.113088 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81082650-c8ea-4a43-b9a1-9b2cc0596d7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=06ab191e97214294853e317575a92daa, user_id=caf0699a2ddf449194e56195e6e5e5c6, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:14.115706 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81082650-c8ea-4a43-b9a1-9b2cc0596d7b admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:14.129166 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1400/2782] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:13:14 2022] PUT /identity/v3/projects/06ab191e97214294853e317575a92daa/users/caf0699a2ddf449194e56195e6e5e5c6/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:13:14.133188 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113313c2-87bd-4997-97c2-e265f4a26a66 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:14.143820 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113313c2-87bd-4997-97c2-e265f4a26a66 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:14.211207 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106273f3-ee6e-4744-9666-4f0a5b63074c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:14.211892 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106273f3-ee6e-4744-9666-4f0a5b63074c admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:14.212174 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106273f3-ee6e-4744-9666-4f0a5b63074c admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:14.212449 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106273f3-ee6e-4744-9666-4f0a5b63074c admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:14.213297 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106273f3-ee6e-4744-9666-4f0a5b63074c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:14.214935 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106273f3-ee6e-4744-9666-4f0a5b63074c admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:14.225822 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1383/2783] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:13:14.230128 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81082650-c8ea-4a43-b9a1-9b2cc0596d7b admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:14.241487 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81082650-c8ea-4a43-b9a1-9b2cc0596d7b admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:14.246075 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af7fafb-a477-444c-b8cd-eccf6a9b3978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:14.248029 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7fafb-a477-444c-b8cd-eccf6a9b3978 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:14.248311 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7fafb-a477-444c-b8cd-eccf6a9b3978 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:14.248585 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7fafb-a477-444c-b8cd-eccf6a9b3978 admin admin] PATH_INFO: `/v3/projects/06ab191e97214294853e317575a92daa/users/caf0699a2ddf449194e56195e6e5e5c6/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:14.250550 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af7fafb-a477-444c-b8cd-eccf6a9b3978 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06ab191e97214294853e317575a92daa, user_id=caf0699a2ddf449194e56195e6e5e5c6, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:14.253181 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af7fafb-a477-444c-b8cd-eccf6a9b3978 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:14.263529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1401/2784] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:13:14 2022] PUT /identity/v3/projects/06ab191e97214294853e317575a92daa/users/caf0699a2ddf449194e56195e6e5e5c6/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 15:13:14.274245 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aca56a-3cff-4558-a6dd-0392b87e3502 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:14.274245 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aca56a-3cff-4558-a6dd-0392b87e3502 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:14.274463 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aca56a-3cff-4558-a6dd-0392b87e3502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:14.301126 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-c1aca56a-3cff-4558-a6dd-0392b87e3502 None None] MFA Rules not processed for user `caf0699a2ddf449194e56195e6e5e5c6`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:13:14.302331 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-c1aca56a-3cff-4558-a6dd-0392b87e3502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:13:14.367852 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1384/2785] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:13:14 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:13:14.379985 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af7fafb-a477-444c-b8cd-eccf6a9b3978 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:14.381890 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-9af7fafb-a477-444c-b8cd-eccf6a9b3978 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:13:14.458816 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af7fafb-a477-444c-b8cd-eccf6a9b3978 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:14.462455 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca982e-1e8b-4c66-bec9-1f458dd6f600 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:14.463116 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca982e-1e8b-4c66-bec9-1f458dd6f600 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:14.463385 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca982e-1e8b-4c66-bec9-1f458dd6f600 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:14.463661 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca982e-1e8b-4c66-bec9-1f458dd6f600 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:14.473437 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ca982e-1e8b-4c66-bec9-1f458dd6f600 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:14.474860 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ca982e-1e8b-4c66-bec9-1f458dd6f600 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:14.485082 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1402/2786] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:13:14 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:13:14.556717 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aca56a-3cff-4558-a6dd-0392b87e3502 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:14.566150 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aca56a-3cff-4558-a6dd-0392b87e3502 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:14.569627 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d1e1ba-37bc-4c77-8d7a-4db56bc9eda3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:14.570261 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1e1ba-37bc-4c77-8d7a-4db56bc9eda3 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:14.570536 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1e1ba-37bc-4c77-8d7a-4db56bc9eda3 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:14.570807 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1e1ba-37bc-4c77-8d7a-4db56bc9eda3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:14.632903 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d1e1ba-37bc-4c77-8d7a-4db56bc9eda3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:14.634278 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d1e1ba-37bc-4c77-8d7a-4db56bc9eda3 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:14.644988 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1385/2787] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:13:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:13:24.598389 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca982e-1e8b-4c66-bec9-1f458dd6f600 service cinder] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:24.662433 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca982e-1e8b-4c66-bec9-1f458dd6f600 service cinder] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:24.667256 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6449f96f-67ae-4ede-b557-fdab8612f48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:24.668652 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449f96f-67ae-4ede-b557-fdab8612f48b admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:24.668942 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449f96f-67ae-4ede-b557-fdab8612f48b admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:24.669219 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449f96f-67ae-4ede-b557-fdab8612f48b admin admin] PATH_INFO: `/v3/users/f32dac66fccf413f8255507011eee5ce` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:24.670426 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6449f96f-67ae-4ede-b557-fdab8612f48b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f32dac66fccf413f8255507011eee5ce)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:24.672349 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6449f96f-67ae-4ede-b557-fdab8612f48b admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:24.718823 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-6449f96f-67ae-4ede-b557-fdab8612f48b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f32dac66fccf413f8255507011eee5ce'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:24.725879 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-6449f96f-67ae-4ede-b557-fdab8612f48b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f32dac66fccf413f8255507011eee5ce'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:24.733618 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-6449f96f-67ae-4ede-b557-fdab8612f48b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f32dac66fccf413f8255507011eee5ce'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:24.734273 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-6449f96f-67ae-4ede-b557-fdab8612f48b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f32dac66fccf413f8255507011eee5ce'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:24.734885 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-6449f96f-67ae-4ede-b557-fdab8612f48b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f32dac66fccf413f8255507011eee5ce'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:24.752328 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1403/2788] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:13:24 2022] DELETE /identity/v3/users/f32dac66fccf413f8255507011eee5ce => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:13:24.756894 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d52ef-ee8c-4864-bb28-9c1eeb0f8a1b None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:24.757398 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d52ef-ee8c-4864-bb28-9c1eeb0f8a1b None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:24.757519 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d52ef-ee8c-4864-bb28-9c1eeb0f8a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:24.774606 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-a99d52ef-ee8c-4864-bb28-9c1eeb0f8a1b None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:13:24.775798 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-a99d52ef-ee8c-4864-bb28-9c1eeb0f8a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:13:24.832573 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1386/2789] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:13:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 15:13:24.842855 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6449f96f-67ae-4ede-b557-fdab8612f48b admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:24.844258 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-6449f96f-67ae-4ede-b557-fdab8612f48b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:13:24.902921 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6449f96f-67ae-4ede-b557-fdab8612f48b admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:24.906853 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd24a022-4fc6-4999-8e1c-4f82ea2d2789 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:24.907505 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd24a022-4fc6-4999-8e1c-4f82ea2d2789 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:24.908044 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd24a022-4fc6-4999-8e1c-4f82ea2d2789 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:24.908322 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd24a022-4fc6-4999-8e1c-4f82ea2d2789 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:24.917605 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd24a022-4fc6-4999-8e1c-4f82ea2d2789 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:24.919726 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd24a022-4fc6-4999-8e1c-4f82ea2d2789 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:24.929111 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1404/2790] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:13:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 15:13:25.205034 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99d52ef-ee8c-4864-bb28-9c1eeb0f8a1b None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:25.206475 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-a99d52ef-ee8c-4864-bb28-9c1eeb0f8a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:13:25.219579 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99d52ef-ee8c-4864-bb28-9c1eeb0f8a1b None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:25.225372 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d5b26-abe4-4fd8-8fab-751efb5a75fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:25.226388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d5b26-abe4-4fd8-8fab-751efb5a75fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:25.226739 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d5b26-abe4-4fd8-8fab-751efb5a75fc admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:25.227023 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d5b26-abe4-4fd8-8fab-751efb5a75fc admin admin] PATH_INFO: `/v3/projects/7844651fb3904bca917af1b37735afa0` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:25.229708 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42d5b26-abe4-4fd8-8fab-751efb5a75fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=7844651fb3904bca917af1b37735afa0)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:25.232617 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42d5b26-abe4-4fd8-8fab-751efb5a75fc admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:25.283022 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-a42d5b26-abe4-4fd8-8fab-751efb5a75fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7844651fb3904bca917af1b37735afa0', 'request_id': 'req-a42d5b26-abe4-4fd8-8fab-751efb5a75fc'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:25.289027 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-a42d5b26-abe4-4fd8-8fab-751efb5a75fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7844651fb3904bca917af1b37735afa0', 'request_id': 'req-a42d5b26-abe4-4fd8-8fab-751efb5a75fc'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:25.297576 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-a42d5b26-abe4-4fd8-8fab-751efb5a75fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7844651fb3904bca917af1b37735afa0', 'request_id': 'req-a42d5b26-abe4-4fd8-8fab-751efb5a75fc'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:25.309632 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-a42d5b26-abe4-4fd8-8fab-751efb5a75fc admin admin] The token cache is being invalidate because project 7844651fb3904bca917af1b37735afa0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:13:25.309918 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-a42d5b26-abe4-4fd8-8fab-751efb5a75fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:25.310617 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-a42d5b26-abe4-4fd8-8fab-751efb5a75fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:25.312178 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1387/2791] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:13:25 2022] DELETE /identity/v3/projects/7844651fb3904bca917af1b37735afa0 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:13:25.316265 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd24a022-4fc6-4999-8e1c-4f82ea2d2789 service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:25.327227 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd24a022-4fc6-4999-8e1c-4f82ea2d2789 service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:25.386353 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea41e9-f2b3-43e5-87dc-08a45bec239d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:25.387356 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea41e9-f2b3-43e5-87dc-08a45bec239d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:25.387632 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea41e9-f2b3-43e5-87dc-08a45bec239d admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:25.387904 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea41e9-f2b3-43e5-87dc-08a45bec239d admin admin] PATH_INFO: `/v3/users/caf0699a2ddf449194e56195e6e5e5c6` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:25.389169 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea41e9-f2b3-43e5-87dc-08a45bec239d admin admin] RBAC: Authorizing `identity:delete_user(user_id=caf0699a2ddf449194e56195e6e5e5c6)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:25.391120 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea41e9-f2b3-43e5-87dc-08a45bec239d admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:25.440156 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d1ea41e9-f2b3-43e5-87dc-08a45bec239d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'caf0699a2ddf449194e56195e6e5e5c6'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:25.446959 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d1ea41e9-f2b3-43e5-87dc-08a45bec239d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'caf0699a2ddf449194e56195e6e5e5c6'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:25.454885 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d1ea41e9-f2b3-43e5-87dc-08a45bec239d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'caf0699a2ddf449194e56195e6e5e5c6'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:25.455627 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d1ea41e9-f2b3-43e5-87dc-08a45bec239d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'caf0699a2ddf449194e56195e6e5e5c6'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:25.456231 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-d1ea41e9-f2b3-43e5-87dc-08a45bec239d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'caf0699a2ddf449194e56195e6e5e5c6'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:25.476940 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1405/2792] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:13:25 2022] DELETE /identity/v3/users/caf0699a2ddf449194e56195e6e5e5c6 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:13:25.760032 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d5b26-abe4-4fd8-8fab-751efb5a75fc admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:25.761522 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-a42d5b26-abe4-4fd8-8fab-751efb5a75fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:13:25.786780 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d5b26-abe4-4fd8-8fab-751efb5a75fc admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:25.850553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ed4c5-ff98-4fe1-aef9-d7168e14e245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:25.851344 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ed4c5-ff98-4fe1-aef9-d7168e14e245 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:25.851624 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ed4c5-ff98-4fe1-aef9-d7168e14e245 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:25.851901 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ed4c5-ff98-4fe1-aef9-d7168e14e245 admin admin] PATH_INFO: `/v3/projects/06ab191e97214294853e317575a92daa` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:25.853118 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ed4c5-ff98-4fe1-aef9-d7168e14e245 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06ab191e97214294853e317575a92daa)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:25.855019 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ed4c5-ff98-4fe1-aef9-d7168e14e245 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:25.902320 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-216ed4c5-ff98-4fe1-aef9-d7168e14e245 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06ab191e97214294853e317575a92daa', 'request_id': 'req-216ed4c5-ff98-4fe1-aef9-d7168e14e245'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:25.908297 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-216ed4c5-ff98-4fe1-aef9-d7168e14e245 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06ab191e97214294853e317575a92daa', 'request_id': 'req-216ed4c5-ff98-4fe1-aef9-d7168e14e245'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:25.916040 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-216ed4c5-ff98-4fe1-aef9-d7168e14e245 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06ab191e97214294853e317575a92daa', 'request_id': 'req-216ed4c5-ff98-4fe1-aef9-d7168e14e245'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:25.928847 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-216ed4c5-ff98-4fe1-aef9-d7168e14e245 admin admin] The token cache is being invalidate because project 06ab191e97214294853e317575a92daa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:13:25.929133 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-216ed4c5-ff98-4fe1-aef9-d7168e14e245 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:25.929792 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-216ed4c5-ff98-4fe1-aef9-d7168e14e245 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:25.931334 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1388/2793] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:13:25 2022] DELETE /identity/v3/projects/06ab191e97214294853e317575a92daa => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:13:25.936827 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365f886-5ade-4f82-ba24-8febf220292a None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:25.937372 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365f886-5ade-4f82-ba24-8febf220292a None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:25.937372 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365f886-5ade-4f82-ba24-8febf220292a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:25.956346 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-7365f886-5ade-4f82-ba24-8febf220292a None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:13:25.957592 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-7365f886-5ade-4f82-ba24-8febf220292a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:13:26.017130 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1406/2794] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:13:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:13:26.025495 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1972aa-ccc2-42c6-b189-8481fbc19f7a None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:26.025780 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1972aa-ccc2-42c6-b189-8481fbc19f7a None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:26.026061 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1972aa-ccc2-42c6-b189-8481fbc19f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:26.045532 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-ae1972aa-ccc2-42c6-b189-8481fbc19f7a None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:13:26.046749 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-ae1972aa-ccc2-42c6-b189-8481fbc19f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:13:26.054880 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1389/2795] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:13:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:13:26.059829 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7365f886-5ade-4f82-ba24-8febf220292a None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:26.069743 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7365f886-5ade-4f82-ba24-8febf220292a None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:26.074493 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7e1a79-89cf-4fda-bb6d-2723cb3433bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:26.075521 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e1a79-89cf-4fda-bb6d-2723cb3433bb admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:26.075807 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e1a79-89cf-4fda-bb6d-2723cb3433bb admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:26.076086 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e1a79-89cf-4fda-bb6d-2723cb3433bb admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:26.076859 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7e1a79-89cf-4fda-bb6d-2723cb3433bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 15:13:26.077166 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7e1a79-89cf-4fda-bb6d-2723cb3433bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:26.078753 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7e1a79-89cf-4fda-bb6d-2723cb3433bb admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:26.091945 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1407/2796] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 15:13:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 15:13:26.096292 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1972aa-ccc2-42c6-b189-8481fbc19f7a None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:26.106212 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1972aa-ccc2-42c6-b189-8481fbc19f7a None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:26.111007 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40604b3a-4139-4e76-b49e-f4d303145743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:26.111661 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40604b3a-4139-4e76-b49e-f4d303145743 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:26.111944 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40604b3a-4139-4e76-b49e-f4d303145743 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:26.112219 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40604b3a-4139-4e76-b49e-f4d303145743 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:26.112951 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40604b3a-4139-4e76-b49e-f4d303145743 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:26.114871 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40604b3a-4139-4e76-b49e-f4d303145743 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:26.142974 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1390/2797] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:13:26 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:13:26.147526 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7e1a79-89cf-4fda-bb6d-2723cb3433bb admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:26.158278 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7e1a79-89cf-4fda-bb6d-2723cb3433bb admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:26.218012 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a8085-05bb-40d1-bcf0-1938fcbd56d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:26.218859 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a8085-05bb-40d1-bcf0-1938fcbd56d3 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:26.219236 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a8085-05bb-40d1-bcf0-1938fcbd56d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:26.219642 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a8085-05bb-40d1-bcf0-1938fcbd56d3 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:26.220510 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114a8085-05bb-40d1-bcf0-1938fcbd56d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:26.222732 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114a8085-05bb-40d1-bcf0-1938fcbd56d3 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:26.251880 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1408/2798] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:13:26 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 15:13:26.256068 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40604b3a-4139-4e76-b49e-f4d303145743 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:26.266318 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40604b3a-4139-4e76-b49e-f4d303145743 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:26.271043 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147fcb9b-dc5e-48bc-b38d-be68860b697b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:26.271703 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147fcb9b-dc5e-48bc-b38d-be68860b697b admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:26.271990 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147fcb9b-dc5e-48bc-b38d-be68860b697b admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:26.272266 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147fcb9b-dc5e-48bc-b38d-be68860b697b admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:26.273022 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147fcb9b-dc5e-48bc-b38d-be68860b697b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:26.274606 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147fcb9b-dc5e-48bc-b38d-be68860b697b admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:26.285906 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1391/2799] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:13:26.290054 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a8085-05bb-40d1-bcf0-1938fcbd56d3 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:26.302197 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a8085-05bb-40d1-bcf0-1938fcbd56d3 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:26.306900 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd9ec68-9b93-4cb5-890f-844227a3dcdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:26.307818 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd9ec68-9b93-4cb5-890f-844227a3dcdc admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:26.307818 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd9ec68-9b93-4cb5-890f-844227a3dcdc admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:26.308113 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd9ec68-9b93-4cb5-890f-844227a3dcdc admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:26.309552 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd9ec68-9b93-4cb5-890f-844227a3dcdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:26.311142 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd9ec68-9b93-4cb5-890f-844227a3dcdc admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:26.321287 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1409/2800] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:13:26.326057 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147fcb9b-dc5e-48bc-b38d-be68860b697b admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:26.336295 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147fcb9b-dc5e-48bc-b38d-be68860b697b admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:26.341348 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b1c7d0-8258-4402-8508-d261311939c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:26.342236 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1c7d0-8258-4402-8508-d261311939c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:26.342524 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1c7d0-8258-4402-8508-d261311939c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:26.342797 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1c7d0-8258-4402-8508-d261311939c0 admin admin] PATH_INFO: `/v3/projects/0c8db37ef1174ce4a8af58192aaa2140/users/c14fbdfc3fe14f8d80d466d15c68b3ce/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:26.359471 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b1c7d0-8258-4402-8508-d261311939c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c8db37ef1174ce4a8af58192aaa2140, user_id=c14fbdfc3fe14f8d80d466d15c68b3ce, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:26.362079 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b1c7d0-8258-4402-8508-d261311939c0 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:26.385075 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1392/2801] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:13:26 2022] PUT /identity/v3/projects/0c8db37ef1174ce4a8af58192aaa2140/users/c14fbdfc3fe14f8d80d466d15c68b3ce/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:13:26.387069 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a60ff4-90c1-4abb-a8b8-ba9c914a43f8 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:26.387934 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a60ff4-90c1-4abb-a8b8-ba9c914a43f8 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:26.388218 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a60ff4-90c1-4abb-a8b8-ba9c914a43f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:26.417146 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-71a60ff4-90c1-4abb-a8b8-ba9c914a43f8 None None] MFA Rules not processed for user `c14fbdfc3fe14f8d80d466d15c68b3ce`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:13:26.419552 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-71a60ff4-90c1-4abb-a8b8-ba9c914a43f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:13:26.489155 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1410/2802] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:13:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:13:26.493621 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b1c7d0-8258-4402-8508-d261311939c0 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:26.557457 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b1c7d0-8258-4402-8508-d261311939c0 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:26.562735 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c48d992-9203-4982-b98d-37fc8ad2a4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:26.563410 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c48d992-9203-4982-b98d-37fc8ad2a4b3 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:26.563694 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c48d992-9203-4982-b98d-37fc8ad2a4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:26.563970 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c48d992-9203-4982-b98d-37fc8ad2a4b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:26.564694 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c48d992-9203-4982-b98d-37fc8ad2a4b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:26.566581 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c48d992-9203-4982-b98d-37fc8ad2a4b3 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:26.593264 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1393/2803] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:13:26 2022] POST /identity/v3/projects => generated 377 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:13:26.597373 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a60ff4-90c1-4abb-a8b8-ba9c914a43f8 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:26.608065 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a60ff4-90c1-4abb-a8b8-ba9c914a43f8 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:26.667925 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9a8426-104f-4cd0-9803-11fa8da08ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:26.668395 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a8426-104f-4cd0-9803-11fa8da08ced admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:26.669342 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a8426-104f-4cd0-9803-11fa8da08ced admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:26.669342 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a8426-104f-4cd0-9803-11fa8da08ced admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:26.669675 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9a8426-104f-4cd0-9803-11fa8da08ced admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:26.671587 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9a8426-104f-4cd0-9803-11fa8da08ced admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:26.693899 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1411/2804] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:13:26 2022] POST /identity/v3/users => generated 295 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:13:26.698852 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c48d992-9203-4982-b98d-37fc8ad2a4b3 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:26.709887 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c48d992-9203-4982-b98d-37fc8ad2a4b3 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:26.714845 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ceae9-0662-4ddd-b422-4e57fb4cc5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:26.715548 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ceae9-0662-4ddd-b422-4e57fb4cc5c1 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:26.715937 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ceae9-0662-4ddd-b422-4e57fb4cc5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:26.716223 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ceae9-0662-4ddd-b422-4e57fb4cc5c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:26.716996 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4ceae9-0662-4ddd-b422-4e57fb4cc5c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:26.718590 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4ceae9-0662-4ddd-b422-4e57fb4cc5c1 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:26.729584 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1394/2805] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:13:26.734194 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9a8426-104f-4cd0-9803-11fa8da08ced admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:26.744404 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9a8426-104f-4cd0-9803-11fa8da08ced admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:26.750111 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc47a838-7ece-407b-ab5d-dd063c8d7035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:26.751038 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47a838-7ece-407b-ab5d-dd063c8d7035 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:26.751321 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47a838-7ece-407b-ab5d-dd063c8d7035 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:26.751599 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47a838-7ece-407b-ab5d-dd063c8d7035 admin admin] PATH_INFO: `/v3/projects/b300f66959864802aaae62c033ef71c1/users/08d3fb0b1ba54bc0bf08a6c839c96cac/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:26.767815 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc47a838-7ece-407b-ab5d-dd063c8d7035 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b300f66959864802aaae62c033ef71c1, user_id=08d3fb0b1ba54bc0bf08a6c839c96cac, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:26.770595 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc47a838-7ece-407b-ab5d-dd063c8d7035 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:26.788145 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1412/2806] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:13:26 2022] PUT /identity/v3/projects/b300f66959864802aaae62c033ef71c1/users/08d3fb0b1ba54bc0bf08a6c839c96cac/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:13:26.792446 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ceae9-0662-4ddd-b422-4e57fb4cc5c1 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:26.803262 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ceae9-0662-4ddd-b422-4e57fb4cc5c1 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:26.860989 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a91986-f1d9-4656-840e-fd70f661ac6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:26.861663 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a91986-f1d9-4656-840e-fd70f661ac6a admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:26.861974 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a91986-f1d9-4656-840e-fd70f661ac6a admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:26.862283 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a91986-f1d9-4656-840e-fd70f661ac6a admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:26.863030 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a91986-f1d9-4656-840e-fd70f661ac6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:26.864649 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a91986-f1d9-4656-840e-fd70f661ac6a admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:26.876093 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1395/2807] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:13:26.880299 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc47a838-7ece-407b-ab5d-dd063c8d7035 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:26.890845 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc47a838-7ece-407b-ab5d-dd063c8d7035 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:26.896097 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403e8ef4-ac7e-471a-98ec-b4bd0501d2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:26.897023 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e8ef4-ac7e-471a-98ec-b4bd0501d2ce admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:26.897312 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e8ef4-ac7e-471a-98ec-b4bd0501d2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:26.897593 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e8ef4-ac7e-471a-98ec-b4bd0501d2ce admin admin] PATH_INFO: `/v3/projects/b300f66959864802aaae62c033ef71c1/users/08d3fb0b1ba54bc0bf08a6c839c96cac/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:26.900005 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403e8ef4-ac7e-471a-98ec-b4bd0501d2ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=b300f66959864802aaae62c033ef71c1, user_id=08d3fb0b1ba54bc0bf08a6c839c96cac, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:26.902815 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403e8ef4-ac7e-471a-98ec-b4bd0501d2ce admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:26.913712 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1413/2808] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:13:26 2022] PUT /identity/v3/projects/b300f66959864802aaae62c033ef71c1/users/08d3fb0b1ba54bc0bf08a6c839c96cac/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:13:26.927417 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d89b3e-9c2e-42c7-9fcf-4e567f574f86 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:26.927729 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d89b3e-9c2e-42c7-9fcf-4e567f574f86 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:26.928022 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d89b3e-9c2e-42c7-9fcf-4e567f574f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:26.956494 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-76d89b3e-9c2e-42c7-9fcf-4e567f574f86 None None] MFA Rules not processed for user `08d3fb0b1ba54bc0bf08a6c839c96cac`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:13:26.957735 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-76d89b3e-9c2e-42c7-9fcf-4e567f574f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:13:27.035612 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1396/2809] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:13:26 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:13:27.048753 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403e8ef4-ac7e-471a-98ec-b4bd0501d2ce admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:27.050454 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-403e8ef4-ac7e-471a-98ec-b4bd0501d2ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:13:27.109614 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403e8ef4-ac7e-471a-98ec-b4bd0501d2ce admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:27.113340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58afd3ed-f2dd-4cdb-8dba-859bfce0f5db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:27.113986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afd3ed-f2dd-4cdb-8dba-859bfce0f5db service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:27.114265 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afd3ed-f2dd-4cdb-8dba-859bfce0f5db service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:27.114536 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afd3ed-f2dd-4cdb-8dba-859bfce0f5db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:27.184162 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58afd3ed-f2dd-4cdb-8dba-859bfce0f5db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:27.185534 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58afd3ed-f2dd-4cdb-8dba-859bfce0f5db service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:27.194673 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1414/2810] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:13:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:13:32.821463 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d89b3e-9c2e-42c7-9fcf-4e567f574f86 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:32.822862 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-76d89b3e-9c2e-42c7-9fcf-4e567f574f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:13:32.835441 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d89b3e-9c2e-42c7-9fcf-4e567f574f86 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:32.838953 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab44f5-fa46-49d9-ade1-2957e0ea7592 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:32.839592 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab44f5-fa46-49d9-ade1-2957e0ea7592 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:32.840482 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab44f5-fa46-49d9-ade1-2957e0ea7592 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:32.840980 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab44f5-fa46-49d9-ade1-2957e0ea7592 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:32.850093 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab44f5-fa46-49d9-ade1-2957e0ea7592 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:32.851521 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab44f5-fa46-49d9-ade1-2957e0ea7592 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:32.861141 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1397/2811] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:13:32 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:13:57.964900 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58afd3ed-f2dd-4cdb-8dba-859bfce0f5db service cinder] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:58.032630 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58afd3ed-f2dd-4cdb-8dba-859bfce0f5db service cinder] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:58.037683 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2dfc39-ba84-48a7-bfd7-b83eda711764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:58.038580 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2dfc39-ba84-48a7-bfd7-b83eda711764 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:58.038861 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2dfc39-ba84-48a7-bfd7-b83eda711764 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:58.039138 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2dfc39-ba84-48a7-bfd7-b83eda711764 admin admin] PATH_INFO: `/v3/users/c14fbdfc3fe14f8d80d466d15c68b3ce` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:58.040478 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2dfc39-ba84-48a7-bfd7-b83eda711764 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c14fbdfc3fe14f8d80d466d15c68b3ce)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:58.043275 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2dfc39-ba84-48a7-bfd7-b83eda711764 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:58.099977 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-0f2dfc39-ba84-48a7-bfd7-b83eda711764 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c14fbdfc3fe14f8d80d466d15c68b3ce'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:58.108906 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-0f2dfc39-ba84-48a7-bfd7-b83eda711764 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c14fbdfc3fe14f8d80d466d15c68b3ce'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:58.117272 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-0f2dfc39-ba84-48a7-bfd7-b83eda711764 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c14fbdfc3fe14f8d80d466d15c68b3ce'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:58.118549 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-0f2dfc39-ba84-48a7-bfd7-b83eda711764 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c14fbdfc3fe14f8d80d466d15c68b3ce'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:58.119326 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-0f2dfc39-ba84-48a7-bfd7-b83eda711764 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c14fbdfc3fe14f8d80d466d15c68b3ce'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:58.140851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1415/2812] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:13:57 2022] DELETE /identity/v3/users/c14fbdfc3fe14f8d80d466d15c68b3ce => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:13:58.145757 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed57828-7f62-4440-958a-4677c3ee002c None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:58.146396 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed57828-7f62-4440-958a-4677c3ee002c None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:58.146396 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed57828-7f62-4440-958a-4677c3ee002c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:58.166871 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-9ed57828-7f62-4440-958a-4677c3ee002c None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:13:58.168855 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-9ed57828-7f62-4440-958a-4677c3ee002c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:13:58.242436 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1398/2813] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:13:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:13:58.258456 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2dfc39-ba84-48a7-bfd7-b83eda711764 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:58.259926 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-0f2dfc39-ba84-48a7-bfd7-b83eda711764 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:13:58.355480 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2dfc39-ba84-48a7-bfd7-b83eda711764 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:58.360105 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3e84df-82b1-4eea-9f48-25df16c8e27d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:58.361003 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e84df-82b1-4eea-9f48-25df16c8e27d service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:58.361340 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e84df-82b1-4eea-9f48-25df16c8e27d service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:58.361625 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e84df-82b1-4eea-9f48-25df16c8e27d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:58.374894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3e84df-82b1-4eea-9f48-25df16c8e27d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:58.376625 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3e84df-82b1-4eea-9f48-25df16c8e27d service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:58.390620 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1416/2814] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:13:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:13:58.695697 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed57828-7f62-4440-958a-4677c3ee002c None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:58.698066 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-9ed57828-7f62-4440-958a-4677c3ee002c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:13:58.716553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed57828-7f62-4440-958a-4677c3ee002c None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:58.731458 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d92181-01a3-4735-ab2a-ec262ac7accc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:58.732345 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d92181-01a3-4735-ab2a-ec262ac7accc admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:58.732651 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d92181-01a3-4735-ab2a-ec262ac7accc admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:58.732929 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d92181-01a3-4735-ab2a-ec262ac7accc admin admin] PATH_INFO: `/v3/projects/0c8db37ef1174ce4a8af58192aaa2140` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:58.734160 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d92181-01a3-4735-ab2a-ec262ac7accc admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c8db37ef1174ce4a8af58192aaa2140)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:58.736060 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d92181-01a3-4735-ab2a-ec262ac7accc admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:58.785403 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-04d92181-01a3-4735-ab2a-ec262ac7accc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c8db37ef1174ce4a8af58192aaa2140', 'request_id': 'req-04d92181-01a3-4735-ab2a-ec262ac7accc'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:58.790694 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-04d92181-01a3-4735-ab2a-ec262ac7accc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c8db37ef1174ce4a8af58192aaa2140', 'request_id': 'req-04d92181-01a3-4735-ab2a-ec262ac7accc'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:58.798026 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-04d92181-01a3-4735-ab2a-ec262ac7accc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c8db37ef1174ce4a8af58192aaa2140', 'request_id': 'req-04d92181-01a3-4735-ab2a-ec262ac7accc'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:58.809385 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-04d92181-01a3-4735-ab2a-ec262ac7accc admin admin] The token cache is being invalidate because project 0c8db37ef1174ce4a8af58192aaa2140 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:13:58.809677 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-04d92181-01a3-4735-ab2a-ec262ac7accc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:58.810343 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-04d92181-01a3-4735-ab2a-ec262ac7accc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:58.811909 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1399/2815] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:13:58 2022] DELETE /identity/v3/projects/0c8db37ef1174ce4a8af58192aaa2140 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:13:58.815955 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3e84df-82b1-4eea-9f48-25df16c8e27d service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:58.827929 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3e84df-82b1-4eea-9f48-25df16c8e27d service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:58.892939 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd509b-3d3f-4b87-8274-ed8e4e48d089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:58.893912 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd509b-3d3f-4b87-8274-ed8e4e48d089 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:58.894192 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd509b-3d3f-4b87-8274-ed8e4e48d089 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:58.894552 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd509b-3d3f-4b87-8274-ed8e4e48d089 admin admin] PATH_INFO: `/v3/users/08d3fb0b1ba54bc0bf08a6c839c96cac` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:58.895796 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddd509b-3d3f-4b87-8274-ed8e4e48d089 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08d3fb0b1ba54bc0bf08a6c839c96cac)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:58.898109 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddd509b-3d3f-4b87-8274-ed8e4e48d089 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:58.955010 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-eddd509b-3d3f-4b87-8274-ed8e4e48d089 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08d3fb0b1ba54bc0bf08a6c839c96cac'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:58.963003 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-eddd509b-3d3f-4b87-8274-ed8e4e48d089 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08d3fb0b1ba54bc0bf08a6c839c96cac'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:58.971138 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-eddd509b-3d3f-4b87-8274-ed8e4e48d089 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08d3fb0b1ba54bc0bf08a6c839c96cac'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:58.971830 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-eddd509b-3d3f-4b87-8274-ed8e4e48d089 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08d3fb0b1ba54bc0bf08a6c839c96cac'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:58.972456 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-eddd509b-3d3f-4b87-8274-ed8e4e48d089 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08d3fb0b1ba54bc0bf08a6c839c96cac'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:58.994231 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1417/2816] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:13:58 2022] DELETE /identity/v3/users/08d3fb0b1ba54bc0bf08a6c839c96cac => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:13:59.293977 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d92181-01a3-4735-ab2a-ec262ac7accc admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:59.295963 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-04d92181-01a3-4735-ab2a-ec262ac7accc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:13:59.311138 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d92181-01a3-4735-ab2a-ec262ac7accc admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:59.384856 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a85c3a5-58bf-4bdc-ab06-e7f9409db2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:59.386706 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a85c3a5-58bf-4bdc-ab06-e7f9409db2e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:59.387224 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a85c3a5-58bf-4bdc-ab06-e7f9409db2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:59.387515 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a85c3a5-58bf-4bdc-ab06-e7f9409db2e9 admin admin] PATH_INFO: `/v3/projects/b300f66959864802aaae62c033ef71c1` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:59.389369 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a85c3a5-58bf-4bdc-ab06-e7f9409db2e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b300f66959864802aaae62c033ef71c1)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:59.393052 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a85c3a5-58bf-4bdc-ab06-e7f9409db2e9 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:59.450895 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-4a85c3a5-58bf-4bdc-ab06-e7f9409db2e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b300f66959864802aaae62c033ef71c1', 'request_id': 'req-4a85c3a5-58bf-4bdc-ab06-e7f9409db2e9'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:59.457157 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-4a85c3a5-58bf-4bdc-ab06-e7f9409db2e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b300f66959864802aaae62c033ef71c1', 'request_id': 'req-4a85c3a5-58bf-4bdc-ab06-e7f9409db2e9'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:59.465020 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-4a85c3a5-58bf-4bdc-ab06-e7f9409db2e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b300f66959864802aaae62c033ef71c1', 'request_id': 'req-4a85c3a5-58bf-4bdc-ab06-e7f9409db2e9'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:59.478108 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-4a85c3a5-58bf-4bdc-ab06-e7f9409db2e9 admin admin] The token cache is being invalidate because project b300f66959864802aaae62c033ef71c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:13:59.478418 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-4a85c3a5-58bf-4bdc-ab06-e7f9409db2e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:59.479214 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-4a85c3a5-58bf-4bdc-ab06-e7f9409db2e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:13:59.481468 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1400/2817] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:13:59 2022] DELETE /identity/v3/projects/b300f66959864802aaae62c033ef71c1 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 15:13:59.487243 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f79ec-7473-4dba-8af4-841f2d10cab0 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:59.487576 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f79ec-7473-4dba-8af4-841f2d10cab0 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:59.487897 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f79ec-7473-4dba-8af4-841f2d10cab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:59.508078 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-6a1f79ec-7473-4dba-8af4-841f2d10cab0 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:13:59.509344 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-6a1f79ec-7473-4dba-8af4-841f2d10cab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:13:59.585879 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1418/2818] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:13:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:13:59.594489 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd7256-f196-4642-ab32-7461ae12d846 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:59.594814 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd7256-f196-4642-ab32-7461ae12d846 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:59.595103 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd7256-f196-4642-ab32-7461ae12d846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:59.615038 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-04bd7256-f196-4642-ab32-7461ae12d846 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:13:59.616240 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-04bd7256-f196-4642-ab32-7461ae12d846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:13:59.626742 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1401/2819] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:13:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:13:59.633174 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1f79ec-7473-4dba-8af4-841f2d10cab0 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:59.645128 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1f79ec-7473-4dba-8af4-841f2d10cab0 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:59.649944 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba79e4-b898-4000-98e3-41ef7ed3fedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:59.650611 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba79e4-b898-4000-98e3-41ef7ed3fedb admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:59.650893 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba79e4-b898-4000-98e3-41ef7ed3fedb admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:59.651177 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba79e4-b898-4000-98e3-41ef7ed3fedb admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:59.652401 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ba79e4-b898-4000-98e3-41ef7ed3fedb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 15:13:59.652760 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ba79e4-b898-4000-98e3-41ef7ed3fedb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:59.654511 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ba79e4-b898-4000-98e3-41ef7ed3fedb admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:59.669368 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1419/2820] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 15:13:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 15:13:59.674095 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd7256-f196-4642-ab32-7461ae12d846 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:59.683975 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd7256-f196-4642-ab32-7461ae12d846 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:59.688996 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f037d-edd1-4891-8562-2fa5ecd0b109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:59.689673 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f037d-edd1-4891-8562-2fa5ecd0b109 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:59.689958 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f037d-edd1-4891-8562-2fa5ecd0b109 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:59.690235 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f037d-edd1-4891-8562-2fa5ecd0b109 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:59.690932 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f037d-edd1-4891-8562-2fa5ecd0b109 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:59.692935 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f037d-edd1-4891-8562-2fa5ecd0b109 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:59.720041 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1402/2821] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:13:59 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 15:13:59.724273 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba79e4-b898-4000-98e3-41ef7ed3fedb admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:59.735520 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba79e4-b898-4000-98e3-41ef7ed3fedb admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:59.796337 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ab5ba-3143-424b-9bf3-1e4743ff06fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:59.797062 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ab5ba-3143-424b-9bf3-1e4743ff06fb admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:59.797343 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ab5ba-3143-424b-9bf3-1e4743ff06fb admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:59.797645 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ab5ba-3143-424b-9bf3-1e4743ff06fb admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:59.798528 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8ab5ba-3143-424b-9bf3-1e4743ff06fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:59.800786 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8ab5ba-3143-424b-9bf3-1e4743ff06fb admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:59.824284 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1420/2822] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:13:59 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:13:59.829175 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f037d-edd1-4891-8562-2fa5ecd0b109 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:59.841209 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f037d-edd1-4891-8562-2fa5ecd0b109 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:59.846400 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27965579-d3e3-4a98-987b-e001c377b1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:59.847137 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27965579-d3e3-4a98-987b-e001c377b1d7 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:59.847425 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27965579-d3e3-4a98-987b-e001c377b1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:59.847704 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27965579-d3e3-4a98-987b-e001c377b1d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:59.848516 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27965579-d3e3-4a98-987b-e001c377b1d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:59.850337 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27965579-d3e3-4a98-987b-e001c377b1d7 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:59.863615 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1403/2823] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:13:59.867671 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ab5ba-3143-424b-9bf3-1e4743ff06fb admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:59.879088 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ab5ba-3143-424b-9bf3-1e4743ff06fb admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:59.888069 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab37c364-5764-4c7c-9382-0f7e7908310e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:59.888991 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab37c364-5764-4c7c-9382-0f7e7908310e admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:59.889411 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab37c364-5764-4c7c-9382-0f7e7908310e admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:59.889786 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab37c364-5764-4c7c-9382-0f7e7908310e admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:59.890851 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab37c364-5764-4c7c-9382-0f7e7908310e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:59.892760 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab37c364-5764-4c7c-9382-0f7e7908310e admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:59.904235 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1421/2824] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:13:59.908698 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27965579-d3e3-4a98-987b-e001c377b1d7 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:13:59.919637 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27965579-d3e3-4a98-987b-e001c377b1d7 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:13:59.924450 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0fee2c-c3f0-49bd-8138-8fd801d45576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:13:59.925398 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0fee2c-c3f0-49bd-8138-8fd801d45576 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:59.925674 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0fee2c-c3f0-49bd-8138-8fd801d45576 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:59.925974 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0fee2c-c3f0-49bd-8138-8fd801d45576 admin admin] PATH_INFO: `/v3/projects/66986bceabc6481a8f4fd14ccd015b92/users/6b441dc24e6841eabdf4f90cc6ff99e9/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:13:59.942083 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0fee2c-c3f0-49bd-8138-8fd801d45576 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66986bceabc6481a8f4fd14ccd015b92, user_id=6b441dc24e6841eabdf4f90cc6ff99e9, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:13:59.944773 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0fee2c-c3f0-49bd-8138-8fd801d45576 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:13:59.958777 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1404/2825] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:13:59 2022] PUT /identity/v3/projects/66986bceabc6481a8f4fd14ccd015b92/users/6b441dc24e6841eabdf4f90cc6ff99e9/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 15:13:59.970546 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5452e88-d26a-427e-8f56-3d1776fc5ede None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:13:59.970839 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5452e88-d26a-427e-8f56-3d1776fc5ede None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:13:59.971130 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5452e88-d26a-427e-8f56-3d1776fc5ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:00.003871 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-f5452e88-d26a-427e-8f56-3d1776fc5ede None None] MFA Rules not processed for user `6b441dc24e6841eabdf4f90cc6ff99e9`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:14:00.005640 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f5452e88-d26a-427e-8f56-3d1776fc5ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:14:00.086886 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1422/2826] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:13:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 15:14:00.091463 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0fee2c-c3f0-49bd-8138-8fd801d45576 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:00.162194 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0fee2c-c3f0-49bd-8138-8fd801d45576 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:00.167994 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e861fe-af27-4918-a1d6-0a250a004078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:00.168851 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e861fe-af27-4918-a1d6-0a250a004078 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:00.169138 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e861fe-af27-4918-a1d6-0a250a004078 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:00.169515 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e861fe-af27-4918-a1d6-0a250a004078 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:00.170483 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e861fe-af27-4918-a1d6-0a250a004078 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:00.172555 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e861fe-af27-4918-a1d6-0a250a004078 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:00.204224 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1405/2827] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:14:00 2022] POST /identity/v3/projects => generated 387 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:14:00.210199 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5452e88-d26a-427e-8f56-3d1776fc5ede None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:00.222523 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5452e88-d26a-427e-8f56-3d1776fc5ede None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:00.280203 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dea6f4-712c-4512-8131-0e79503d6c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:00.280912 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dea6f4-712c-4512-8131-0e79503d6c44 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:00.281191 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dea6f4-712c-4512-8131-0e79503d6c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:00.281491 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dea6f4-712c-4512-8131-0e79503d6c44 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:00.282163 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dea6f4-712c-4512-8131-0e79503d6c44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:00.284310 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dea6f4-712c-4512-8131-0e79503d6c44 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:00.309857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1423/2828] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:14:00 2022] POST /identity/v3/users => generated 300 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 15:14:00.314920 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e861fe-af27-4918-a1d6-0a250a004078 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:00.328501 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e861fe-af27-4918-a1d6-0a250a004078 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:00.333470 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4e422-71aa-4a66-9568-4363e3d88cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:00.334157 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4e422-71aa-4a66-9568-4363e3d88cef admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:00.334444 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4e422-71aa-4a66-9568-4363e3d88cef admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:00.334720 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4e422-71aa-4a66-9568-4363e3d88cef admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:00.335456 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f4e422-71aa-4a66-9568-4363e3d88cef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:00.337144 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f4e422-71aa-4a66-9568-4363e3d88cef admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:00.355682 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1406/2829] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:14:00.359869 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dea6f4-712c-4512-8131-0e79503d6c44 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:00.370057 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dea6f4-712c-4512-8131-0e79503d6c44 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:00.375247 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9b307a-5435-41df-b553-508fbd0c7434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:00.376133 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b307a-5435-41df-b553-508fbd0c7434 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:00.376420 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b307a-5435-41df-b553-508fbd0c7434 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:00.376728 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b307a-5435-41df-b553-508fbd0c7434 admin admin] PATH_INFO: `/v3/projects/5cbcded8967c4a0b927ef3a6418e2ec1/users/0bb6a3cbb84a4f19a5ed91344f87f8a3/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:00.395879 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9b307a-5435-41df-b553-508fbd0c7434 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cbcded8967c4a0b927ef3a6418e2ec1, user_id=0bb6a3cbb84a4f19a5ed91344f87f8a3, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:00.398710 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9b307a-5435-41df-b553-508fbd0c7434 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:00.416555 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1424/2830] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:14:00 2022] PUT /identity/v3/projects/5cbcded8967c4a0b927ef3a6418e2ec1/users/0bb6a3cbb84a4f19a5ed91344f87f8a3/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:14:00.421722 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4e422-71aa-4a66-9568-4363e3d88cef admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:00.438870 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4e422-71aa-4a66-9568-4363e3d88cef admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:00.522032 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0001ae8-7229-4a4e-9239-30827a384f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:00.523080 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0001ae8-7229-4a4e-9239-30827a384f5c admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:00.523457 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0001ae8-7229-4a4e-9239-30827a384f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:00.523799 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0001ae8-7229-4a4e-9239-30827a384f5c admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:00.524585 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0001ae8-7229-4a4e-9239-30827a384f5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:00.527106 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0001ae8-7229-4a4e-9239-30827a384f5c admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:00.538295 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1407/2831] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:14:00.544156 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9b307a-5435-41df-b553-508fbd0c7434 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:00.556291 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9b307a-5435-41df-b553-508fbd0c7434 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:00.561433 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb0e9d-6e2e-4f5b-b625-91e7dc0cee66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:00.562337 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb0e9d-6e2e-4f5b-b625-91e7dc0cee66 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:00.562622 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb0e9d-6e2e-4f5b-b625-91e7dc0cee66 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:00.562922 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb0e9d-6e2e-4f5b-b625-91e7dc0cee66 admin admin] PATH_INFO: `/v3/projects/5cbcded8967c4a0b927ef3a6418e2ec1/users/0bb6a3cbb84a4f19a5ed91344f87f8a3/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:00.565254 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fb0e9d-6e2e-4f5b-b625-91e7dc0cee66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cbcded8967c4a0b927ef3a6418e2ec1, user_id=0bb6a3cbb84a4f19a5ed91344f87f8a3, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:00.567989 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fb0e9d-6e2e-4f5b-b625-91e7dc0cee66 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:00.581456 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1425/2832] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:14:00 2022] PUT /identity/v3/projects/5cbcded8967c4a0b927ef3a6418e2ec1/users/0bb6a3cbb84a4f19a5ed91344f87f8a3/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 15:14:00.592678 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff3401-bb29-4e70-b226-5d556387d02a None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:00.593068 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff3401-bb29-4e70-b226-5d556387d02a None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:00.593445 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff3401-bb29-4e70-b226-5d556387d02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:00.622191 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-35ff3401-bb29-4e70-b226-5d556387d02a None None] MFA Rules not processed for user `0bb6a3cbb84a4f19a5ed91344f87f8a3`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:14:00.623414 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-35ff3401-bb29-4e70-b226-5d556387d02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:14:00.707387 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1408/2833] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:14:00 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 15:14:00.721708 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb0e9d-6e2e-4f5b-b625-91e7dc0cee66 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:00.723601 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-b2fb0e9d-6e2e-4f5b-b625-91e7dc0cee66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:14:00.783986 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb0e9d-6e2e-4f5b-b625-91e7dc0cee66 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:00.787659 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f26cb0-445b-4ca9-9cf8-e22b96f26d59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:00.788313 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f26cb0-445b-4ca9-9cf8-e22b96f26d59 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:00.788593 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f26cb0-445b-4ca9-9cf8-e22b96f26d59 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:00.788903 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f26cb0-445b-4ca9-9cf8-e22b96f26d59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:00.799151 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f26cb0-445b-4ca9-9cf8-e22b96f26d59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:00.800690 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f26cb0-445b-4ca9-9cf8-e22b96f26d59 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:00.810710 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1426/2834] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:14:00 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:14:00.840194 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff3401-bb29-4e70-b226-5d556387d02a None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:00.906631 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff3401-bb29-4e70-b226-5d556387d02a None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:00.911440 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee79d5-d01a-475e-ba78-6c7c91f1853d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:00.912311 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee79d5-d01a-475e-ba78-6c7c91f1853d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:00.912585 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee79d5-d01a-475e-ba78-6c7c91f1853d admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:00.912919 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee79d5-d01a-475e-ba78-6c7c91f1853d admin admin] PATH_INFO: `/v3/users/6b441dc24e6841eabdf4f90cc6ff99e9` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:00.914092 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ee79d5-d01a-475e-ba78-6c7c91f1853d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b441dc24e6841eabdf4f90cc6ff99e9)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:00.915997 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ee79d5-d01a-475e-ba78-6c7c91f1853d admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:00.968019 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-22ee79d5-d01a-475e-ba78-6c7c91f1853d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b441dc24e6841eabdf4f90cc6ff99e9'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:00.968646 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-22ee79d5-d01a-475e-ba78-6c7c91f1853d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b441dc24e6841eabdf4f90cc6ff99e9'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:00.977308 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-22ee79d5-d01a-475e-ba78-6c7c91f1853d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b441dc24e6841eabdf4f90cc6ff99e9'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:00.986512 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-22ee79d5-d01a-475e-ba78-6c7c91f1853d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b441dc24e6841eabdf4f90cc6ff99e9'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:00.987351 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-22ee79d5-d01a-475e-ba78-6c7c91f1853d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b441dc24e6841eabdf4f90cc6ff99e9'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:01.007293 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1409/2835] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:14:00 2022] DELETE /identity/v3/users/6b441dc24e6841eabdf4f90cc6ff99e9 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:14:01.013244 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3c9f4-06d2-43ff-b840-2b5ef2759f4a None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:01.013794 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3c9f4-06d2-43ff-b840-2b5ef2759f4a None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:01.013916 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3c9f4-06d2-43ff-b840-2b5ef2759f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:01.033087 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-59e3c9f4-06d2-43ff-b840-2b5ef2759f4a None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:14:01.034302 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-59e3c9f4-06d2-43ff-b840-2b5ef2759f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:14:01.097564 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1427/2836] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:14:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:14:01.110047 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee79d5-d01a-475e-ba78-6c7c91f1853d admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:01.112314 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-22ee79d5-d01a-475e-ba78-6c7c91f1853d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:14:01.194628 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee79d5-d01a-475e-ba78-6c7c91f1853d admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:01.199561 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e6e3b7-e486-4b57-a2ef-2538c6e20119 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:01.200404 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6e3b7-e486-4b57-a2ef-2538c6e20119 service neutron] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:01.200719 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6e3b7-e486-4b57-a2ef-2538c6e20119 service neutron] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:01.201098 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6e3b7-e486-4b57-a2ef-2538c6e20119 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:01.212364 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e6e3b7-e486-4b57-a2ef-2538c6e20119 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:01.214166 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e6e3b7-e486-4b57-a2ef-2538c6e20119 service neutron] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:01.226416 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1410/2837] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:14:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:14:01.530801 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e3c9f4-06d2-43ff-b840-2b5ef2759f4a None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:01.532455 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-59e3c9f4-06d2-43ff-b840-2b5ef2759f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:14:01.546006 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e3c9f4-06d2-43ff-b840-2b5ef2759f4a None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:01.551941 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81dc2b5-d8b1-4ecd-8feb-4041f04f0a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:01.553567 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81dc2b5-d8b1-4ecd-8feb-4041f04f0a31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:01.554250 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81dc2b5-d8b1-4ecd-8feb-4041f04f0a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:01.554794 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81dc2b5-d8b1-4ecd-8feb-4041f04f0a31 admin admin] PATH_INFO: `/v3/projects/66986bceabc6481a8f4fd14ccd015b92` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:01.557184 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81dc2b5-d8b1-4ecd-8feb-4041f04f0a31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66986bceabc6481a8f4fd14ccd015b92)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:01.562580 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81dc2b5-d8b1-4ecd-8feb-4041f04f0a31 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:01.616789 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b81dc2b5-d8b1-4ecd-8feb-4041f04f0a31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66986bceabc6481a8f4fd14ccd015b92', 'request_id': 'req-b81dc2b5-d8b1-4ecd-8feb-4041f04f0a31'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:01.623358 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b81dc2b5-d8b1-4ecd-8feb-4041f04f0a31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66986bceabc6481a8f4fd14ccd015b92', 'request_id': 'req-b81dc2b5-d8b1-4ecd-8feb-4041f04f0a31'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:01.630598 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b81dc2b5-d8b1-4ecd-8feb-4041f04f0a31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66986bceabc6481a8f4fd14ccd015b92', 'request_id': 'req-b81dc2b5-d8b1-4ecd-8feb-4041f04f0a31'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:01.643312 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b81dc2b5-d8b1-4ecd-8feb-4041f04f0a31 admin admin] The token cache is being invalidate because project 66986bceabc6481a8f4fd14ccd015b92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:14:01.643604 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b81dc2b5-d8b1-4ecd-8feb-4041f04f0a31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:01.644263 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-b81dc2b5-d8b1-4ecd-8feb-4041f04f0a31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:01.645838 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1428/2838] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:14:01 2022] DELETE /identity/v3/projects/66986bceabc6481a8f4fd14ccd015b92 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 15:14:01.650153 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e6e3b7-e486-4b57-a2ef-2538c6e20119 service neutron] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:01.661198 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e6e3b7-e486-4b57-a2ef-2538c6e20119 service neutron] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:01.723051 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa376a8-e745-439b-b565-3d4cd1ed434d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:01.724103 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa376a8-e745-439b-b565-3d4cd1ed434d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:01.724578 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa376a8-e745-439b-b565-3d4cd1ed434d admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:01.725166 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa376a8-e745-439b-b565-3d4cd1ed434d admin admin] PATH_INFO: `/v3/users/0bb6a3cbb84a4f19a5ed91344f87f8a3` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:01.726826 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa376a8-e745-439b-b565-3d4cd1ed434d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bb6a3cbb84a4f19a5ed91344f87f8a3)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:01.729168 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa376a8-e745-439b-b565-3d4cd1ed434d admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:01.793905 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-eaa376a8-e745-439b-b565-3d4cd1ed434d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bb6a3cbb84a4f19a5ed91344f87f8a3'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:01.794518 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-eaa376a8-e745-439b-b565-3d4cd1ed434d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bb6a3cbb84a4f19a5ed91344f87f8a3'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:01.803595 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-eaa376a8-e745-439b-b565-3d4cd1ed434d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bb6a3cbb84a4f19a5ed91344f87f8a3'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:01.812950 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-eaa376a8-e745-439b-b565-3d4cd1ed434d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bb6a3cbb84a4f19a5ed91344f87f8a3'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:01.813720 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-eaa376a8-e745-439b-b565-3d4cd1ed434d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bb6a3cbb84a4f19a5ed91344f87f8a3'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:01.832295 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1411/2839] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:14:01 2022] DELETE /identity/v3/users/0bb6a3cbb84a4f19a5ed91344f87f8a3 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:14:02.153939 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81dc2b5-d8b1-4ecd-8feb-4041f04f0a31 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:02.155490 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-b81dc2b5-d8b1-4ecd-8feb-4041f04f0a31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:14:02.172123 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81dc2b5-d8b1-4ecd-8feb-4041f04f0a31 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:02.238702 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c05cd-e88f-4831-be1b-51cbc5d02494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:02.239606 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c05cd-e88f-4831-be1b-51cbc5d02494 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:02.239920 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c05cd-e88f-4831-be1b-51cbc5d02494 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:02.240206 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c05cd-e88f-4831-be1b-51cbc5d02494 admin admin] PATH_INFO: `/v3/projects/5cbcded8967c4a0b927ef3a6418e2ec1` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:02.241495 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3c05cd-e88f-4831-be1b-51cbc5d02494 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cbcded8967c4a0b927ef3a6418e2ec1)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:02.243530 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3c05cd-e88f-4831-be1b-51cbc5d02494 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:02.297318 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-de3c05cd-e88f-4831-be1b-51cbc5d02494 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cbcded8967c4a0b927ef3a6418e2ec1', 'request_id': 'req-de3c05cd-e88f-4831-be1b-51cbc5d02494'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:02.304629 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-de3c05cd-e88f-4831-be1b-51cbc5d02494 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cbcded8967c4a0b927ef3a6418e2ec1', 'request_id': 'req-de3c05cd-e88f-4831-be1b-51cbc5d02494'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:02.314257 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-de3c05cd-e88f-4831-be1b-51cbc5d02494 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cbcded8967c4a0b927ef3a6418e2ec1', 'request_id': 'req-de3c05cd-e88f-4831-be1b-51cbc5d02494'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:02.327897 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-de3c05cd-e88f-4831-be1b-51cbc5d02494 admin admin] The token cache is being invalidate because project 5cbcded8967c4a0b927ef3a6418e2ec1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79888) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:14:02.328191 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-de3c05cd-e88f-4831-be1b-51cbc5d02494 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:02.329029 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-de3c05cd-e88f-4831-be1b-51cbc5d02494 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:02.330655 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1429/2840] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:14:02 2022] DELETE /identity/v3/projects/5cbcded8967c4a0b927ef3a6418e2ec1 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:14:02.336735 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b38f0dd-ffb8-437a-be79-21fbab187584 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:02.337075 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b38f0dd-ffb8-437a-be79-21fbab187584 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:02.337398 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b38f0dd-ffb8-437a-be79-21fbab187584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:02.358514 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-9b38f0dd-ffb8-437a-be79-21fbab187584 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:14:02.359728 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-9b38f0dd-ffb8-437a-be79-21fbab187584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:14:02.429525 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1412/2841] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:14:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:14:02.438373 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2adafa-eb7e-484d-97ea-a84698e0024b None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:02.438658 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2adafa-eb7e-484d-97ea-a84698e0024b None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:02.438941 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2adafa-eb7e-484d-97ea-a84698e0024b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:02.459236 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-cf2adafa-eb7e-484d-97ea-a84698e0024b None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:14:02.460442 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-cf2adafa-eb7e-484d-97ea-a84698e0024b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:14:02.469317 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1430/2842] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:14:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:14:02.473712 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b38f0dd-ffb8-437a-be79-21fbab187584 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:02.483685 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b38f0dd-ffb8-437a-be79-21fbab187584 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:02.488635 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95efbc9f-8b53-4a31-a368-7da1d816ba36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:02.489305 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95efbc9f-8b53-4a31-a368-7da1d816ba36 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:02.489584 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95efbc9f-8b53-4a31-a368-7da1d816ba36 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:02.489864 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95efbc9f-8b53-4a31-a368-7da1d816ba36 admin admin] PATH_INFO: `/v3/domains` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:02.490677 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95efbc9f-8b53-4a31-a368-7da1d816ba36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79887) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 15:14:02.491029 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95efbc9f-8b53-4a31-a368-7da1d816ba36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:02.492819 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95efbc9f-8b53-4a31-a368-7da1d816ba36 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:02.505792 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1413/2843] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 15:14:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 15:14:02.510501 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2adafa-eb7e-484d-97ea-a84698e0024b None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:02.522657 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2adafa-eb7e-484d-97ea-a84698e0024b None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:02.529532 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f7a74-796a-47db-8cd2-2ba705598fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:02.530234 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f7a74-796a-47db-8cd2-2ba705598fef admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:02.530654 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f7a74-796a-47db-8cd2-2ba705598fef admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:02.531076 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f7a74-796a-47db-8cd2-2ba705598fef admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:02.532047 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f7a74-796a-47db-8cd2-2ba705598fef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:02.535119 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f7a74-796a-47db-8cd2-2ba705598fef admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:02.567301 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1431/2844] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:14:02 2022] POST /identity/v3/projects => generated 389 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:14:02.571544 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95efbc9f-8b53-4a31-a368-7da1d816ba36 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:02.582954 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95efbc9f-8b53-4a31-a368-7da1d816ba36 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:02.650784 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de949d-4cb8-4a17-b47b-66cdf61bc24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:02.651707 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de949d-4cb8-4a17-b47b-66cdf61bc24e admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:02.651881 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de949d-4cb8-4a17-b47b-66cdf61bc24e admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:02.652058 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de949d-4cb8-4a17-b47b-66cdf61bc24e admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:02.652720 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23de949d-4cb8-4a17-b47b-66cdf61bc24e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:02.654936 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23de949d-4cb8-4a17-b47b-66cdf61bc24e admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:02.681656 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1414/2845] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:14:02 2022] POST /identity/v3/users => generated 295 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:14:02.687198 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f7a74-796a-47db-8cd2-2ba705598fef admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:02.708180 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f7a74-796a-47db-8cd2-2ba705598fef admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:02.713470 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad619a5-4ca6-4832-80cb-39761777d496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:02.714968 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad619a5-4ca6-4832-80cb-39761777d496 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:02.715284 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad619a5-4ca6-4832-80cb-39761777d496 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:02.715631 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad619a5-4ca6-4832-80cb-39761777d496 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:02.716502 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad619a5-4ca6-4832-80cb-39761777d496 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:02.718444 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad619a5-4ca6-4832-80cb-39761777d496 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:02.730403 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1432/2846] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 06 15:14:02.734456 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de949d-4cb8-4a17-b47b-66cdf61bc24e admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:02.744761 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de949d-4cb8-4a17-b47b-66cdf61bc24e admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:02.750486 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa41ad8-1c4a-4a47-b262-3a9d1c35f026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:02.751261 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa41ad8-1c4a-4a47-b262-3a9d1c35f026 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:02.751543 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa41ad8-1c4a-4a47-b262-3a9d1c35f026 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:02.751822 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa41ad8-1c4a-4a47-b262-3a9d1c35f026 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:02.752541 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa41ad8-1c4a-4a47-b262-3a9d1c35f026 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:02.754308 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa41ad8-1c4a-4a47-b262-3a9d1c35f026 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:02.765874 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1415/2847] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:14:02.770727 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad619a5-4ca6-4832-80cb-39761777d496 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:02.781373 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad619a5-4ca6-4832-80cb-39761777d496 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:02.786596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de094b90-959c-4b56-a3cc-088d18b4f1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:02.787495 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de094b90-959c-4b56-a3cc-088d18b4f1c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:02.787769 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de094b90-959c-4b56-a3cc-088d18b4f1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:02.788046 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de094b90-959c-4b56-a3cc-088d18b4f1c6 admin admin] PATH_INFO: `/v3/projects/77d4c54396f04f24b6eb99e10d942043/users/850bd7a42cd14e8a8e8fe1450e8a4454/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:02.804636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de094b90-959c-4b56-a3cc-088d18b4f1c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77d4c54396f04f24b6eb99e10d942043, user_id=850bd7a42cd14e8a8e8fe1450e8a4454, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:02.807542 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de094b90-959c-4b56-a3cc-088d18b4f1c6 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:02.822595 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1433/2848] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:14:02 2022] PUT /identity/v3/projects/77d4c54396f04f24b6eb99e10d942043/users/850bd7a42cd14e8a8e8fe1450e8a4454/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 15:14:02.889753 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db210f20-2c73-497e-a11a-f1d84086b566 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:02.890076 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db210f20-2c73-497e-a11a-f1d84086b566 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:02.890299 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db210f20-2c73-497e-a11a-f1d84086b566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:02.920536 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-db210f20-2c73-497e-a11a-f1d84086b566 None None] MFA Rules not processed for user `850bd7a42cd14e8a8e8fe1450e8a4454`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:14:02.921804 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-db210f20-2c73-497e-a11a-f1d84086b566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:14:02.988140 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1416/2849] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:14:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 15:14:02.993212 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de094b90-959c-4b56-a3cc-088d18b4f1c6 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:03.062641 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de094b90-959c-4b56-a3cc-088d18b4f1c6 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:03.067879 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d0909-9403-4cf3-bed7-82ceb9547590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:03.068649 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d0909-9403-4cf3-bed7-82ceb9547590 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:03.068939 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d0909-9403-4cf3-bed7-82ceb9547590 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:03.069351 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d0909-9403-4cf3-bed7-82ceb9547590 admin admin] PATH_INFO: `/v3/projects` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:03.070129 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d0909-9403-4cf3-bed7-82ceb9547590 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:03.072264 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d0909-9403-4cf3-bed7-82ceb9547590 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:03.102557 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1434/2850] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:14:02 2022] POST /identity/v3/projects => generated 389 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 15:14:03.107679 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db210f20-2c73-497e-a11a-f1d84086b566 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:03.126962 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db210f20-2c73-497e-a11a-f1d84086b566 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:03.196107 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a8e36-d4b2-471b-9196-0ea643743b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:03.196831 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a8e36-d4b2-471b-9196-0ea643743b2e admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:03.197119 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a8e36-d4b2-471b-9196-0ea643743b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:03.197398 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a8e36-d4b2-471b-9196-0ea643743b2e admin admin] PATH_INFO: `/v3/users` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:03.198101 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273a8e36-d4b2-471b-9196-0ea643743b2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:03.200466 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273a8e36-d4b2-471b-9196-0ea643743b2e admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:03.223833 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1417/2851] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:14:03 2022] POST /identity/v3/users => generated 301 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:14:03.228471 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d0909-9403-4cf3-bed7-82ceb9547590 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:03.244591 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d0909-9403-4cf3-bed7-82ceb9547590 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:03.249891 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805242aa-aa8a-4b6a-afed-9dd2fa8eb03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:03.250579 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805242aa-aa8a-4b6a-afed-9dd2fa8eb03b admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:03.250868 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805242aa-aa8a-4b6a-afed-9dd2fa8eb03b admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:03.251158 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805242aa-aa8a-4b6a-afed-9dd2fa8eb03b admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:03.251894 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805242aa-aa8a-4b6a-afed-9dd2fa8eb03b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:03.253583 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805242aa-aa8a-4b6a-afed-9dd2fa8eb03b admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:03.266030 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1435/2852] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:14:03 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:14:03.271065 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a8e36-d4b2-471b-9196-0ea643743b2e admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:03.282734 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a8e36-d4b2-471b-9196-0ea643743b2e admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:03.288386 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f581089a-452f-41c2-89e1-941283560bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:03.289373 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f581089a-452f-41c2-89e1-941283560bc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:03.289659 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f581089a-452f-41c2-89e1-941283560bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:03.289936 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f581089a-452f-41c2-89e1-941283560bc0 admin admin] PATH_INFO: `/v3/projects/b30e2648e97a417884491475226da76e/users/a5d76eee57d046818ab32029fcae89af/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:03.307380 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f581089a-452f-41c2-89e1-941283560bc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b30e2648e97a417884491475226da76e, user_id=a5d76eee57d046818ab32029fcae89af, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:03.310374 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f581089a-452f-41c2-89e1-941283560bc0 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:03.325487 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1418/2853] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:14:03 2022] PUT /identity/v3/projects/b30e2648e97a417884491475226da76e/users/a5d76eee57d046818ab32029fcae89af/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:14:03.329962 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805242aa-aa8a-4b6a-afed-9dd2fa8eb03b admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:03.341917 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805242aa-aa8a-4b6a-afed-9dd2fa8eb03b admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:03.410109 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15abbd1f-643a-4f23-8e6a-8e79be2fc5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:03.410778 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abbd1f-643a-4f23-8e6a-8e79be2fc5db admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:03.411065 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abbd1f-643a-4f23-8e6a-8e79be2fc5db admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:03.411450 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abbd1f-643a-4f23-8e6a-8e79be2fc5db admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:03.412197 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15abbd1f-643a-4f23-8e6a-8e79be2fc5db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:03.413931 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15abbd1f-643a-4f23-8e6a-8e79be2fc5db admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:03.427325 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1436/2854] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:14:03 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:14:03.433388 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f581089a-452f-41c2-89e1-941283560bc0 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:03.445082 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f581089a-452f-41c2-89e1-941283560bc0 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:03.450787 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb501661-ed0c-4f9d-8748-19f773960345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:03.451760 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb501661-ed0c-4f9d-8748-19f773960345 admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:03.452108 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb501661-ed0c-4f9d-8748-19f773960345 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:03.452392 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb501661-ed0c-4f9d-8748-19f773960345 admin admin] PATH_INFO: `/v3/projects/b30e2648e97a417884491475226da76e/users/a5d76eee57d046818ab32029fcae89af/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:03.454624 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb501661-ed0c-4f9d-8748-19f773960345 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b30e2648e97a417884491475226da76e, user_id=a5d76eee57d046818ab32029fcae89af, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:03.457815 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb501661-ed0c-4f9d-8748-19f773960345 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:03.470331 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1419/2855] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:14:03 2022] PUT /identity/v3/projects/b30e2648e97a417884491475226da76e/users/a5d76eee57d046818ab32029fcae89af/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:14:03.481706 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef52f66-701e-4f05-8d1f-fdb221bd28bf None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:03.482068 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef52f66-701e-4f05-8d1f-fdb221bd28bf None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:03.482348 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef52f66-701e-4f05-8d1f-fdb221bd28bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:03.511596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-3ef52f66-701e-4f05-8d1f-fdb221bd28bf None None] MFA Rules not processed for user `a5d76eee57d046818ab32029fcae89af`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:14:03.512984 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-3ef52f66-701e-4f05-8d1f-fdb221bd28bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:14:03.606224 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1437/2856] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:14:03 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 15:14:03.617920 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb501661-ed0c-4f9d-8748-19f773960345 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:03.619633 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-cb501661-ed0c-4f9d-8748-19f773960345 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:14:03.680298 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb501661-ed0c-4f9d-8748-19f773960345 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:03.684297 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cab166-ac70-4783-861f-82e70944d2df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:03.685114 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cab166-ac70-4783-861f-82e70944d2df service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:03.685361 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cab166-ac70-4783-861f-82e70944d2df service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:03.685680 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cab166-ac70-4783-861f-82e70944d2df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:03.695581 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cab166-ac70-4783-861f-82e70944d2df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:03.697060 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cab166-ac70-4783-861f-82e70944d2df service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:03.707761 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1420/2857] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:14:03 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 06 15:14:03.856995 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef52f66-701e-4f05-8d1f-fdb221bd28bf None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:03.921795 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef52f66-701e-4f05-8d1f-fdb221bd28bf None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:03.929600 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ec0e7-a1b8-4598-98a1-558b730072f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:03.931053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ec0e7-a1b8-4598-98a1-558b730072f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:03.931377 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ec0e7-a1b8-4598-98a1-558b730072f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:03.931662 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ec0e7-a1b8-4598-98a1-558b730072f6 admin admin] PATH_INFO: `/v3/users/850bd7a42cd14e8a8e8fe1450e8a4454` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:03.933521 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2ec0e7-a1b8-4598-98a1-558b730072f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=850bd7a42cd14e8a8e8fe1450e8a4454)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:03.935428 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2ec0e7-a1b8-4598-98a1-558b730072f6 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:03.984114 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-6d2ec0e7-a1b8-4598-98a1-558b730072f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '850bd7a42cd14e8a8e8fe1450e8a4454'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:03.992560 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-6d2ec0e7-a1b8-4598-98a1-558b730072f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '850bd7a42cd14e8a8e8fe1450e8a4454'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:04.002596 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-6d2ec0e7-a1b8-4598-98a1-558b730072f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '850bd7a42cd14e8a8e8fe1450e8a4454'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:04.003478 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-6d2ec0e7-a1b8-4598-98a1-558b730072f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '850bd7a42cd14e8a8e8fe1450e8a4454'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:04.004212 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-6d2ec0e7-a1b8-4598-98a1-558b730072f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '850bd7a42cd14e8a8e8fe1450e8a4454'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:04.024130 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1438/2858] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:14:03 2022] DELETE /identity/v3/users/850bd7a42cd14e8a8e8fe1450e8a4454 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:14:04.029157 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b9c56-cb65-4963-92d1-ac676607afbe None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:04.029452 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b9c56-cb65-4963-92d1-ac676607afbe None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:04.029729 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b9c56-cb65-4963-92d1-ac676607afbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:04.048815 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-cc9b9c56-cb65-4963-92d1-ac676607afbe None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:14:04.050268 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-cc9b9c56-cb65-4963-92d1-ac676607afbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:14:04.110856 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1421/2859] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:14:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:14:04.121245 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ec0e7-a1b8-4598-98a1-558b730072f6 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:04.122670 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-6d2ec0e7-a1b8-4598-98a1-558b730072f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:14:04.184629 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ec0e7-a1b8-4598-98a1-558b730072f6 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:04.188374 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77331f2-e2d2-4e30-8a7f-bd6efab21a6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:04.189165 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77331f2-e2d2-4e30-8a7f-bd6efab21a6f service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:04.189462 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77331f2-e2d2-4e30-8a7f-bd6efab21a6f service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:04.189732 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77331f2-e2d2-4e30-8a7f-bd6efab21a6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:04.202649 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77331f2-e2d2-4e30-8a7f-bd6efab21a6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:04.204110 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77331f2-e2d2-4e30-8a7f-bd6efab21a6f service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:04.214861 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1439/2860] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:14:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:14:04.539503 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b9c56-cb65-4963-92d1-ac676607afbe None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:04.542564 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-cc9b9c56-cb65-4963-92d1-ac676607afbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:14:04.563787 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b9c56-cb65-4963-92d1-ac676607afbe None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:04.572053 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb595725-a135-4659-9a67-1e70dae174bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:04.573504 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb595725-a135-4659-9a67-1e70dae174bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:04.574001 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb595725-a135-4659-9a67-1e70dae174bc admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:04.574502 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb595725-a135-4659-9a67-1e70dae174bc admin admin] PATH_INFO: `/v3/projects/77d4c54396f04f24b6eb99e10d942043` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:04.576724 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb595725-a135-4659-9a67-1e70dae174bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=77d4c54396f04f24b6eb99e10d942043)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:04.579681 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb595725-a135-4659-9a67-1e70dae174bc admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:04.645040 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-eb595725-a135-4659-9a67-1e70dae174bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77d4c54396f04f24b6eb99e10d942043', 'request_id': 'req-eb595725-a135-4659-9a67-1e70dae174bc'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:04.653707 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-eb595725-a135-4659-9a67-1e70dae174bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77d4c54396f04f24b6eb99e10d942043', 'request_id': 'req-eb595725-a135-4659-9a67-1e70dae174bc'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:04.662213 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-eb595725-a135-4659-9a67-1e70dae174bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77d4c54396f04f24b6eb99e10d942043', 'request_id': 'req-eb595725-a135-4659-9a67-1e70dae174bc'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:04.676219 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-eb595725-a135-4659-9a67-1e70dae174bc admin admin] The token cache is being invalidate because project 77d4c54396f04f24b6eb99e10d942043 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:14:04.676565 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-eb595725-a135-4659-9a67-1e70dae174bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:04.677383 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-eb595725-a135-4659-9a67-1e70dae174bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:04.679345 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1422/2861] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:14:04 2022] DELETE /identity/v3/projects/77d4c54396f04f24b6eb99e10d942043 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:14:04.684024 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77331f2-e2d2-4e30-8a7f-bd6efab21a6f service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:04.695756 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77331f2-e2d2-4e30-8a7f-bd6efab21a6f service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:04.763413 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f024e0a-696a-4c32-b036-6addac22b300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:04.764520 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f024e0a-696a-4c32-b036-6addac22b300 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:04.764942 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f024e0a-696a-4c32-b036-6addac22b300 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:04.765363 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f024e0a-696a-4c32-b036-6addac22b300 admin admin] PATH_INFO: `/v3/users/a5d76eee57d046818ab32029fcae89af` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:04.766842 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f024e0a-696a-4c32-b036-6addac22b300 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5d76eee57d046818ab32029fcae89af)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:04.768900 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f024e0a-696a-4c32-b036-6addac22b300 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:04.819943 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-0f024e0a-696a-4c32-b036-6addac22b300 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5d76eee57d046818ab32029fcae89af'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:04.829467 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-0f024e0a-696a-4c32-b036-6addac22b300 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5d76eee57d046818ab32029fcae89af'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:04.837689 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-0f024e0a-696a-4c32-b036-6addac22b300 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5d76eee57d046818ab32029fcae89af'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:04.838502 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-0f024e0a-696a-4c32-b036-6addac22b300 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5d76eee57d046818ab32029fcae89af'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:04.839735 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-0f024e0a-696a-4c32-b036-6addac22b300 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5d76eee57d046818ab32029fcae89af'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:04.859374 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1440/2862] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:14:04 2022] DELETE /identity/v3/users/a5d76eee57d046818ab32029fcae89af => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 15:14:05.160837 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb595725-a135-4659-9a67-1e70dae174bc admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:05.161590 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-eb595725-a135-4659-9a67-1e70dae174bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:14:05.177999 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb595725-a135-4659-9a67-1e70dae174bc admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:05.239981 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e986855-55b9-473f-9cd6-734385e8a69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:05.240866 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e986855-55b9-473f-9cd6-734385e8a69d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:05.241153 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e986855-55b9-473f-9cd6-734385e8a69d admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:05.241433 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e986855-55b9-473f-9cd6-734385e8a69d admin admin] PATH_INFO: `/v3/projects/b30e2648e97a417884491475226da76e` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:05.242664 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e986855-55b9-473f-9cd6-734385e8a69d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b30e2648e97a417884491475226da76e)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:05.244622 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e986855-55b9-473f-9cd6-734385e8a69d admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:05.297080 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-9e986855-55b9-473f-9cd6-734385e8a69d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b30e2648e97a417884491475226da76e', 'request_id': 'req-9e986855-55b9-473f-9cd6-734385e8a69d'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:05.302942 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-9e986855-55b9-473f-9cd6-734385e8a69d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b30e2648e97a417884491475226da76e', 'request_id': 'req-9e986855-55b9-473f-9cd6-734385e8a69d'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:05.311880 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-9e986855-55b9-473f-9cd6-734385e8a69d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b30e2648e97a417884491475226da76e', 'request_id': 'req-9e986855-55b9-473f-9cd6-734385e8a69d'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:05.325362 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-9e986855-55b9-473f-9cd6-734385e8a69d admin admin] The token cache is being invalidate because project b30e2648e97a417884491475226da76e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:14:05.325635 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-9e986855-55b9-473f-9cd6-734385e8a69d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:05.326325 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-9e986855-55b9-473f-9cd6-734385e8a69d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:05.327925 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1423/2863] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:14:05 2022] DELETE /identity/v3/projects/b30e2648e97a417884491475226da76e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:14:05.333439 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150fd65-f3eb-4000-af14-c1e99c29ea2a None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:05.333732 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150fd65-f3eb-4000-af14-c1e99c29ea2a None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:05.334053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150fd65-f3eb-4000-af14-c1e99c29ea2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:05.358979 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-1150fd65-f3eb-4000-af14-c1e99c29ea2a None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:14:05.360384 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-1150fd65-f3eb-4000-af14-c1e99c29ea2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:14:05.429530 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1441/2864] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:14:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:14:05.438951 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76cff9e-b00f-417a-8628-e54bec456fa8 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:05.439281 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76cff9e-b00f-417a-8628-e54bec456fa8 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:05.439504 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76cff9e-b00f-417a-8628-e54bec456fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:05.462953 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-e76cff9e-b00f-417a-8628-e54bec456fa8 None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:14:05.465155 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-e76cff9e-b00f-417a-8628-e54bec456fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:14:05.474071 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1424/2865] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:14:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:14:05.478627 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1150fd65-f3eb-4000-af14-c1e99c29ea2a None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:05.489198 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1150fd65-f3eb-4000-af14-c1e99c29ea2a None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:05.494105 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bcbce5-527f-4cd8-8365-1e0807997429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:05.494769 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bcbce5-527f-4cd8-8365-1e0807997429 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:05.495970 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bcbce5-527f-4cd8-8365-1e0807997429 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:05.496261 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bcbce5-527f-4cd8-8365-1e0807997429 admin admin] PATH_INFO: `/v3/domains` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:05.497033 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bcbce5-527f-4cd8-8365-1e0807997429 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79888) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 06 15:14:05.497347 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bcbce5-527f-4cd8-8365-1e0807997429 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:05.498982 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bcbce5-527f-4cd8-8365-1e0807997429 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:05.518753 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1442/2866] 10.222.0.23 () {58 vars in 1152 bytes} [Fri May 6 15:14:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 06 15:14:05.523572 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76cff9e-b00f-417a-8628-e54bec456fa8 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:05.534365 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76cff9e-b00f-417a-8628-e54bec456fa8 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:05.539813 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dec7b0-b274-4189-99aa-1a4fbb099c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:05.540492 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dec7b0-b274-4189-99aa-1a4fbb099c4d admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:05.540817 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dec7b0-b274-4189-99aa-1a4fbb099c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:05.541097 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dec7b0-b274-4189-99aa-1a4fbb099c4d admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:05.542100 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dec7b0-b274-4189-99aa-1a4fbb099c4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:05.544292 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dec7b0-b274-4189-99aa-1a4fbb099c4d admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:05.575626 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1425/2867] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:14:05 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 15:14:05.579768 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bcbce5-527f-4cd8-8365-1e0807997429 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:05.589956 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bcbce5-527f-4cd8-8365-1e0807997429 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:05.659507 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ba8aa-49ac-4f5c-9df6-afadf001b629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:05.660415 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ba8aa-49ac-4f5c-9df6-afadf001b629 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:05.660776 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ba8aa-49ac-4f5c-9df6-afadf001b629 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:05.661056 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ba8aa-49ac-4f5c-9df6-afadf001b629 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:05.662168 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7ba8aa-49ac-4f5c-9df6-afadf001b629 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:05.664582 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7ba8aa-49ac-4f5c-9df6-afadf001b629 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:05.688235 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1443/2868] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:14:05 2022] POST /identity/v3/users => generated 290 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 06 15:14:05.692742 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dec7b0-b274-4189-99aa-1a4fbb099c4d admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:05.704558 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dec7b0-b274-4189-99aa-1a4fbb099c4d admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:05.709693 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910701ba-43c5-426d-babd-5501885f761f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:05.710381 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910701ba-43c5-426d-babd-5501885f761f admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:05.710746 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910701ba-43c5-426d-babd-5501885f761f admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:05.710986 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910701ba-43c5-426d-babd-5501885f761f admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:05.711655 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910701ba-43c5-426d-babd-5501885f761f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:05.713567 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910701ba-43c5-426d-babd-5501885f761f admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:05.726530 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1426/2869] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:14:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:14:05.730723 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ba8aa-49ac-4f5c-9df6-afadf001b629 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:05.741424 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ba8aa-49ac-4f5c-9df6-afadf001b629 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:05.747292 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecddf88-f2f9-4394-8920-90da8b215b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:05.747970 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecddf88-f2f9-4394-8920-90da8b215b43 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:05.748259 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecddf88-f2f9-4394-8920-90da8b215b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:05.748536 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecddf88-f2f9-4394-8920-90da8b215b43 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:05.749299 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecddf88-f2f9-4394-8920-90da8b215b43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:05.750942 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecddf88-f2f9-4394-8920-90da8b215b43 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:05.763251 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1444/2870] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:14:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:14:05.767965 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910701ba-43c5-426d-babd-5501885f761f admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:05.779929 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910701ba-43c5-426d-babd-5501885f761f admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:05.786310 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f57ce5f-bc89-48e0-bad7-09eba2e6fbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:05.787408 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57ce5f-bc89-48e0-bad7-09eba2e6fbad admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:05.787712 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57ce5f-bc89-48e0-bad7-09eba2e6fbad admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:05.788070 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57ce5f-bc89-48e0-bad7-09eba2e6fbad admin admin] PATH_INFO: `/v3/projects/29a96297680d4c6db0c55484ec970537/users/55d62fc502a24a87b8f1d13c4e0f11e1/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:05.806923 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f57ce5f-bc89-48e0-bad7-09eba2e6fbad admin admin] RBAC: Authorizing `identity:create_grant(project_id=29a96297680d4c6db0c55484ec970537, user_id=55d62fc502a24a87b8f1d13c4e0f11e1, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:05.810927 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f57ce5f-bc89-48e0-bad7-09eba2e6fbad admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:05.828429 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1427/2871] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:14:05 2022] PUT /identity/v3/projects/29a96297680d4c6db0c55484ec970537/users/55d62fc502a24a87b8f1d13c4e0f11e1/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:14:05.839478 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8cdc80-5616-4c7f-9bfe-9d747a4cc320 None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:05.839855 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8cdc80-5616-4c7f-9bfe-9d747a4cc320 None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:05.840043 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8cdc80-5616-4c7f-9bfe-9d747a4cc320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:05.869652 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-cc8cdc80-5616-4c7f-9bfe-9d747a4cc320 None None] MFA Rules not processed for user `55d62fc502a24a87b8f1d13c4e0f11e1`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:14:05.872004 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-cc8cdc80-5616-4c7f-9bfe-9d747a4cc320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:14:05.949484 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1445/2872] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:14:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:14:05.959098 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f57ce5f-bc89-48e0-bad7-09eba2e6fbad admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:06.040266 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f57ce5f-bc89-48e0-bad7-09eba2e6fbad admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:06.046492 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88f89ee-a3f0-4599-a613-521e4e9d2894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:06.047165 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f89ee-a3f0-4599-a613-521e4e9d2894 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:06.047450 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f89ee-a3f0-4599-a613-521e4e9d2894 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:06.047740 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f89ee-a3f0-4599-a613-521e4e9d2894 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:06.048635 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88f89ee-a3f0-4599-a613-521e4e9d2894 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:06.050601 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88f89ee-a3f0-4599-a613-521e4e9d2894 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:06.079125 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1428/2873] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:14:05 2022] POST /identity/v3/projects => generated 379 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:14:06.083858 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8cdc80-5616-4c7f-9bfe-9d747a4cc320 None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:06.096539 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8cdc80-5616-4c7f-9bfe-9d747a4cc320 None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:06.162812 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0082d800-e4a6-435c-b232-d91b8b0f6280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:06.163937 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0082d800-e4a6-435c-b232-d91b8b0f6280 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:06.164327 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0082d800-e4a6-435c-b232-d91b8b0f6280 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:06.164792 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0082d800-e4a6-435c-b232-d91b8b0f6280 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:06.165979 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0082d800-e4a6-435c-b232-d91b8b0f6280 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:06.169958 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0082d800-e4a6-435c-b232-d91b8b0f6280 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:06.190825 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1446/2874] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:14:06 2022] POST /identity/v3/users => generated 290 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:14:06.195573 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88f89ee-a3f0-4599-a613-521e4e9d2894 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:06.207451 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88f89ee-a3f0-4599-a613-521e4e9d2894 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:06.212975 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8db54e-b7c7-4f19-b8e0-06634eb65a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:06.214198 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8db54e-b7c7-4f19-b8e0-06634eb65a12 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:06.214597 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8db54e-b7c7-4f19-b8e0-06634eb65a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:06.214989 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8db54e-b7c7-4f19-b8e0-06634eb65a12 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:06.216530 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8db54e-b7c7-4f19-b8e0-06634eb65a12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:06.217879 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8db54e-b7c7-4f19-b8e0-06634eb65a12 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:06.231949 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1429/2875] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 06 15:14:06.236162 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0082d800-e4a6-435c-b232-d91b8b0f6280 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:06.247646 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0082d800-e4a6-435c-b232-d91b8b0f6280 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:06.254554 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030946c5-310e-4073-a60e-cf6ce41eafb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:06.255232 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030946c5-310e-4073-a60e-cf6ce41eafb2 admin admin] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:06.255524 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030946c5-310e-4073-a60e-cf6ce41eafb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:06.255857 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030946c5-310e-4073-a60e-cf6ce41eafb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:06.256514 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030946c5-310e-4073-a60e-cf6ce41eafb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:06.258148 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030946c5-310e-4073-a60e-cf6ce41eafb2 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:06.269612 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1447/2876] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 06 15:14:06.274365 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8db54e-b7c7-4f19-b8e0-06634eb65a12 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:06.289982 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8db54e-b7c7-4f19-b8e0-06634eb65a12 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:06.296052 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c552fe-6762-4e36-ab8b-a26262dc3eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:06.296995 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c552fe-6762-4e36-ab8b-a26262dc3eba admin admin] REQUEST_METHOD: `PUT` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:06.297256 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c552fe-6762-4e36-ab8b-a26262dc3eba admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:06.297531 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c552fe-6762-4e36-ab8b-a26262dc3eba admin admin] PATH_INFO: `/v3/projects/f0efd54b77e743ceae9b716a9d301936/users/98b49baee9974d8e89155c57e3b54998/roles/5f76e2e820b64d14b893c679427f4bed` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:06.314144 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c552fe-6762-4e36-ab8b-a26262dc3eba admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0efd54b77e743ceae9b716a9d301936, user_id=98b49baee9974d8e89155c57e3b54998, role_id=5f76e2e820b64d14b893c679427f4bed)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:06.316818 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c552fe-6762-4e36-ab8b-a26262dc3eba admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:06.331786 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1430/2877] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:14:06 2022] PUT /identity/v3/projects/f0efd54b77e743ceae9b716a9d301936/users/98b49baee9974d8e89155c57e3b54998/roles/5f76e2e820b64d14b893c679427f4bed => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:14:06.342373 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f289243-fde2-4fdd-b210-b49e6fe8346a None None] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:06.342788 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f289243-fde2-4fdd-b210-b49e6fe8346a None None] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:06.343076 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f289243-fde2-4fdd-b210-b49e6fe8346a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:06.373777 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.auth.core [None req-8f289243-fde2-4fdd-b210-b49e6fe8346a None None] MFA Rules not processed for user `98b49baee9974d8e89155c57e3b54998`. Rule list: `[]` (Enabled: `True`). {{(pid=79888) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:14:06.374991 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-8f289243-fde2-4fdd-b210-b49e6fe8346a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:14:06.453844 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1448/2878] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:14:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 06 15:14:06.458668 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c552fe-6762-4e36-ab8b-a26262dc3eba admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:06.526620 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c552fe-6762-4e36-ab8b-a26262dc3eba admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:06.531780 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee56acf9-66f9-41a3-b0aa-4cc64d5db467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:06.532458 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56acf9-66f9-41a3-b0aa-4cc64d5db467 admin admin] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:06.532777 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56acf9-66f9-41a3-b0aa-4cc64d5db467 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:06.533321 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56acf9-66f9-41a3-b0aa-4cc64d5db467 admin admin] PATH_INFO: `/v3/projects` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:06.534033 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee56acf9-66f9-41a3-b0aa-4cc64d5db467 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:06.536387 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee56acf9-66f9-41a3-b0aa-4cc64d5db467 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:06.566031 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1431/2879] 10.222.0.23 () {60 vars in 1152 bytes} [Fri May 6 15:14:06 2022] POST /identity/v3/projects => generated 377 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:14:06.570410 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f289243-fde2-4fdd-b210-b49e6fe8346a None None] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:06.583421 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f289243-fde2-4fdd-b210-b49e6fe8346a None None] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:06.653903 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0952329c-76b9-47d8-9587-cedf5d111802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:06.654799 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952329c-76b9-47d8-9587-cedf5d111802 admin admin] REQUEST_METHOD: `POST` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:06.655186 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952329c-76b9-47d8-9587-cedf5d111802 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:06.655570 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952329c-76b9-47d8-9587-cedf5d111802 admin admin] PATH_INFO: `/v3/users` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:06.656479 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0952329c-76b9-47d8-9587-cedf5d111802 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:06.659112 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0952329c-76b9-47d8-9587-cedf5d111802 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:06.680289 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1449/2880] 10.222.0.23 () {60 vars in 1143 bytes} [Fri May 6 15:14:06 2022] POST /identity/v3/users => generated 295 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 06 15:14:06.685375 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee56acf9-66f9-41a3-b0aa-4cc64d5db467 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:06.696264 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee56acf9-66f9-41a3-b0aa-4cc64d5db467 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:06.702553 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a9a7f8-c51d-45ab-9a79-97453cdb8ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:06.703329 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9a7f8-c51d-45ab-9a79-97453cdb8ad0 admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:06.703742 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9a7f8-c51d-45ab-9a79-97453cdb8ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:06.704075 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9a7f8-c51d-45ab-9a79-97453cdb8ad0 admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:06.704890 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a9a7f8-c51d-45ab-9a79-97453cdb8ad0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:06.706621 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a9a7f8-c51d-45ab-9a79-97453cdb8ad0 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:06.722058 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1432/2881] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 06 15:14:06.726991 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0952329c-76b9-47d8-9587-cedf5d111802 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:06.741444 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0952329c-76b9-47d8-9587-cedf5d111802 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:06.748691 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587c335-2652-49a4-be38-279094f8aca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:06.749940 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587c335-2652-49a4-be38-279094f8aca9 admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:06.750312 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587c335-2652-49a4-be38-279094f8aca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:06.750696 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587c335-2652-49a4-be38-279094f8aca9 admin admin] PATH_INFO: `/v3/projects/b3729a0fe92f482481bcf0b4cff9cb1e/users/947fe2053da04a749e49712ac7b87d66/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:06.770758 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d587c335-2652-49a4-be38-279094f8aca9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3729a0fe92f482481bcf0b4cff9cb1e, user_id=947fe2053da04a749e49712ac7b87d66, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:06.774111 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d587c335-2652-49a4-be38-279094f8aca9 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:06.789799 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1450/2882] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:14:06 2022] PUT /identity/v3/projects/b3729a0fe92f482481bcf0b4cff9cb1e/users/947fe2053da04a749e49712ac7b87d66/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:14:06.795612 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a9a7f8-c51d-45ab-9a79-97453cdb8ad0 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:06.806655 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a9a7f8-c51d-45ab-9a79-97453cdb8ad0 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:06.865839 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f363c8ee-036b-4032-9cfe-ec9b585b823f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:06.866431 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363c8ee-036b-4032-9cfe-ec9b585b823f admin admin] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:06.866714 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363c8ee-036b-4032-9cfe-ec9b585b823f admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:06.866992 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363c8ee-036b-4032-9cfe-ec9b585b823f admin admin] PATH_INFO: `/v3/roles` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:06.867719 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f363c8ee-036b-4032-9cfe-ec9b585b823f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:06.869422 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f363c8ee-036b-4032-9cfe-ec9b585b823f admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:06.881059 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1433/2883] 10.222.0.23 () {58 vars in 1121 bytes} [Fri May 6 15:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 06 15:14:06.886226 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587c335-2652-49a4-be38-279094f8aca9 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:06.897372 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587c335-2652-49a4-be38-279094f8aca9 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:06.903181 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5afcf93-8b50-4c7d-9f5d-d9bc2efa86dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:06.904207 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5afcf93-8b50-4c7d-9f5d-d9bc2efa86dc admin admin] REQUEST_METHOD: `PUT` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:06.904498 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5afcf93-8b50-4c7d-9f5d-d9bc2efa86dc admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:06.904812 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5afcf93-8b50-4c7d-9f5d-d9bc2efa86dc admin admin] PATH_INFO: `/v3/projects/b3729a0fe92f482481bcf0b4cff9cb1e/users/947fe2053da04a749e49712ac7b87d66/roles/6dcd35d1a83748759d6ef9f026077835` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:06.906849 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5afcf93-8b50-4c7d-9f5d-d9bc2efa86dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3729a0fe92f482481bcf0b4cff9cb1e, user_id=947fe2053da04a749e49712ac7b87d66, role_id=6dcd35d1a83748759d6ef9f026077835)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:06.910271 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5afcf93-8b50-4c7d-9f5d-d9bc2efa86dc admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:06.923571 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1451/2884] 10.222.0.23 () {60 vars in 1482 bytes} [Fri May 6 15:14:06 2022] PUT /identity/v3/projects/b3729a0fe92f482481bcf0b4cff9cb1e/users/947fe2053da04a749e49712ac7b87d66/roles/6dcd35d1a83748759d6ef9f026077835 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:14:06.943306 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c075a76-5759-45c9-80f2-6f5ae7f944f4 None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:06.943674 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c075a76-5759-45c9-80f2-6f5ae7f944f4 None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:06.943956 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c075a76-5759-45c9-80f2-6f5ae7f944f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:06.981365 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-9c075a76-5759-45c9-80f2-6f5ae7f944f4 None None] MFA Rules not processed for user `947fe2053da04a749e49712ac7b87d66`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:14:06.982981 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-9c075a76-5759-45c9-80f2-6f5ae7f944f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:14:07.064093 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1434/2885] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:14:06 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:14:07.077056 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5afcf93-8b50-4c7d-9f5d-d9bc2efa86dc admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:07.078937 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-f5afcf93-8b50-4c7d-9f5d-d9bc2efa86dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:14:07.153003 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5afcf93-8b50-4c7d-9f5d-d9bc2efa86dc admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:07.160108 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0abd04-6c39-4bba-bb26-8eea1b44bcd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:07.160910 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0abd04-6c39-4bba-bb26-8eea1b44bcd3 service cinder] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:07.161300 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0abd04-6c39-4bba-bb26-8eea1b44bcd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:07.162269 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0abd04-6c39-4bba-bb26-8eea1b44bcd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:07.173604 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0abd04-6c39-4bba-bb26-8eea1b44bcd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:07.175063 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0abd04-6c39-4bba-bb26-8eea1b44bcd3 service cinder] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:07.186486 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1452/2886] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:14:07 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:14:07.239823 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c075a76-5759-45c9-80f2-6f5ae7f944f4 None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:07.241887 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-9c075a76-5759-45c9-80f2-6f5ae7f944f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:14:07.260226 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c075a76-5759-45c9-80f2-6f5ae7f944f4 None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:07.265310 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830ec71b-be75-4d4f-a515-0def839d5c47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6409b68b4832421ba590d7bc53131f7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:07.265979 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ec71b-be75-4d4f-a515-0def839d5c47 service cinder] REQUEST_METHOD: `GET` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:07.266293 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ec71b-be75-4d4f-a515-0def839d5c47 service cinder] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:07.266571 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ec71b-be75-4d4f-a515-0def839d5c47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:07.327671 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830ec71b-be75-4d4f-a515-0def839d5c47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:07.330933 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830ec71b-be75-4d4f-a515-0def839d5c47 service cinder] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:07.348077 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1435/2887] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:14:07 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:14:10.278918 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0abd04-6c39-4bba-bb26-8eea1b44bcd3 service cinder] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:10.341074 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0abd04-6c39-4bba-bb26-8eea1b44bcd3 service cinder] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:10.345862 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631191ef-3cc8-41e2-af9b-42edf020b950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:10.346771 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631191ef-3cc8-41e2-af9b-42edf020b950 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:10.347053 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631191ef-3cc8-41e2-af9b-42edf020b950 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:10.347327 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631191ef-3cc8-41e2-af9b-42edf020b950 admin admin] PATH_INFO: `/v3/users/55d62fc502a24a87b8f1d13c4e0f11e1` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:10.348625 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631191ef-3cc8-41e2-af9b-42edf020b950 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55d62fc502a24a87b8f1d13c4e0f11e1)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:10.351046 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631191ef-3cc8-41e2-af9b-42edf020b950 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:10.400708 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-631191ef-3cc8-41e2-af9b-42edf020b950 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55d62fc502a24a87b8f1d13c4e0f11e1'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:10.409380 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-631191ef-3cc8-41e2-af9b-42edf020b950 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55d62fc502a24a87b8f1d13c4e0f11e1'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:10.416951 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-631191ef-3cc8-41e2-af9b-42edf020b950 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55d62fc502a24a87b8f1d13c4e0f11e1'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:10.417642 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-631191ef-3cc8-41e2-af9b-42edf020b950 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55d62fc502a24a87b8f1d13c4e0f11e1'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:10.418315 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-631191ef-3cc8-41e2-af9b-42edf020b950 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55d62fc502a24a87b8f1d13c4e0f11e1'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:10.436840 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1453/2888] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:14:10 2022] DELETE /identity/v3/users/55d62fc502a24a87b8f1d13c4e0f11e1 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:14:10.441813 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd4e98-2209-4367-bac2-e480171ee78a None None] REQUEST_METHOD: `POST` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:10.442593 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd4e98-2209-4367-bac2-e480171ee78a None None] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:10.442793 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd4e98-2209-4367-bac2-e480171ee78a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:10.460887 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.auth.core [None req-e5dd4e98-2209-4367-bac2-e480171ee78a None None] MFA Rules not processed for user `66f1687219494ee0a402084110e075fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79887) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 06 15:14:10.462087 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-e5dd4e98-2209-4367-bac2-e480171ee78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:14:10.519653 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1436/2889] 10.222.0.23 () {58 vars in 957 bytes} [Fri May 6 15:14:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 06 15:14:10.530200 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631191ef-3cc8-41e2-af9b-42edf020b950 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:10.531637 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-631191ef-3cc8-41e2-af9b-42edf020b950 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:14:10.597809 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631191ef-3cc8-41e2-af9b-42edf020b950 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:10.601610 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b48ed-635f-41d8-b922-47528fec38b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dcd1542d8f643c4996142a88b9b6abb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:10.602247 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b48ed-635f-41d8-b922-47528fec38b7 service neutron] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:10.602529 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b48ed-635f-41d8-b922-47528fec38b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:10.602815 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b48ed-635f-41d8-b922-47528fec38b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:10.612214 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05b48ed-635f-41d8-b922-47528fec38b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:10.613678 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05b48ed-635f-41d8-b922-47528fec38b7 service neutron] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:10.623978 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1454/2890] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:14:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 06 15:14:10.897566 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dd4e98-2209-4367-bac2-e480171ee78a None None] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:10.899169 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-e5dd4e98-2209-4367-bac2-e480171ee78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:14:10.914741 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dd4e98-2209-4367-bac2-e480171ee78a None None] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:10.921845 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e71c5f-e9c3-49e7-8877-b15cfcb7bff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:10.923565 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e71c5f-e9c3-49e7-8877-b15cfcb7bff3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:10.924214 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e71c5f-e9c3-49e7-8877-b15cfcb7bff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:10.924506 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e71c5f-e9c3-49e7-8877-b15cfcb7bff3 admin admin] PATH_INFO: `/v3/projects/29a96297680d4c6db0c55484ec970537` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:10.925749 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e71c5f-e9c3-49e7-8877-b15cfcb7bff3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29a96297680d4c6db0c55484ec970537)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:10.927686 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e71c5f-e9c3-49e7-8877-b15cfcb7bff3 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:10.977234 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-90e71c5f-e9c3-49e7-8877-b15cfcb7bff3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29a96297680d4c6db0c55484ec970537', 'request_id': 'req-90e71c5f-e9c3-49e7-8877-b15cfcb7bff3'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:10.982701 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-90e71c5f-e9c3-49e7-8877-b15cfcb7bff3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29a96297680d4c6db0c55484ec970537', 'request_id': 'req-90e71c5f-e9c3-49e7-8877-b15cfcb7bff3'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:10.991274 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-90e71c5f-e9c3-49e7-8877-b15cfcb7bff3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29a96297680d4c6db0c55484ec970537', 'request_id': 'req-90e71c5f-e9c3-49e7-8877-b15cfcb7bff3'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:11.003915 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-90e71c5f-e9c3-49e7-8877-b15cfcb7bff3 admin admin] The token cache is being invalidate because project 29a96297680d4c6db0c55484ec970537 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:14:11.004201 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-90e71c5f-e9c3-49e7-8877-b15cfcb7bff3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:11.004887 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-90e71c5f-e9c3-49e7-8877-b15cfcb7bff3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:11.006417 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1437/2891] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:14:10 2022] DELETE /identity/v3/projects/29a96297680d4c6db0c55484ec970537 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:14:11.011580 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b48ed-635f-41d8-b922-47528fec38b7 service neutron] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:11.021690 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b48ed-635f-41d8-b922-47528fec38b7 service neutron] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:11.078169 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc31d1f-59c9-4e60-84f6-9c99ebc65163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:11.079043 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc31d1f-59c9-4e60-84f6-9c99ebc65163 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:11.079323 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc31d1f-59c9-4e60-84f6-9c99ebc65163 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:11.079599 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc31d1f-59c9-4e60-84f6-9c99ebc65163 admin admin] PATH_INFO: `/v3/users/98b49baee9974d8e89155c57e3b54998` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:11.080799 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc31d1f-59c9-4e60-84f6-9c99ebc65163 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98b49baee9974d8e89155c57e3b54998)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:11.082717 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc31d1f-59c9-4e60-84f6-9c99ebc65163 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:11.129578 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-1dc31d1f-59c9-4e60-84f6-9c99ebc65163 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98b49baee9974d8e89155c57e3b54998'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:11.137283 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-1dc31d1f-59c9-4e60-84f6-9c99ebc65163 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98b49baee9974d8e89155c57e3b54998'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:11.145166 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-1dc31d1f-59c9-4e60-84f6-9c99ebc65163 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98b49baee9974d8e89155c57e3b54998'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:11.145824 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-1dc31d1f-59c9-4e60-84f6-9c99ebc65163 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98b49baee9974d8e89155c57e3b54998'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:11.146416 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-1dc31d1f-59c9-4e60-84f6-9c99ebc65163 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98b49baee9974d8e89155c57e3b54998'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:11.165231 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1455/2892] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:14:11 2022] DELETE /identity/v3/users/98b49baee9974d8e89155c57e3b54998 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 06 15:14:11.466240 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e71c5f-e9c3-49e7-8877-b15cfcb7bff3 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:11.468505 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-90e71c5f-e9c3-49e7-8877-b15cfcb7bff3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:14:11.484048 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e71c5f-e9c3-49e7-8877-b15cfcb7bff3 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:11.540432 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627dac2-c606-4e71-84bc-d600faa70098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:11.541240 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627dac2-c606-4e71-84bc-d600faa70098 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:11.541516 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627dac2-c606-4e71-84bc-d600faa70098 admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:11.542418 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627dac2-c606-4e71-84bc-d600faa70098 admin admin] PATH_INFO: `/v3/projects/f0efd54b77e743ceae9b716a9d301936` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:11.543694 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d627dac2-c606-4e71-84bc-d600faa70098 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0efd54b77e743ceae9b716a9d301936)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:11.545654 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d627dac2-c606-4e71-84bc-d600faa70098 admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:11.591865 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d627dac2-c606-4e71-84bc-d600faa70098 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0efd54b77e743ceae9b716a9d301936', 'request_id': 'req-d627dac2-c606-4e71-84bc-d600faa70098'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:11.597109 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d627dac2-c606-4e71-84bc-d600faa70098 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0efd54b77e743ceae9b716a9d301936', 'request_id': 'req-d627dac2-c606-4e71-84bc-d600faa70098'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:11.605106 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d627dac2-c606-4e71-84bc-d600faa70098 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0efd54b77e743ceae9b716a9d301936', 'request_id': 'req-d627dac2-c606-4e71-84bc-d600faa70098'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:11.617473 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d627dac2-c606-4e71-84bc-d600faa70098 admin admin] The token cache is being invalidate because project f0efd54b77e743ceae9b716a9d301936 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:14:11.617758 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d627dac2-c606-4e71-84bc-d600faa70098 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:11.618406 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-d627dac2-c606-4e71-84bc-d600faa70098 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:11.620033 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1438/2893] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:14:11 2022] DELETE /identity/v3/projects/f0efd54b77e743ceae9b716a9d301936 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:14:11.624208 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc31d1f-59c9-4e60-84f6-9c99ebc65163 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:11.625777 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-1dc31d1f-59c9-4e60-84f6-9c99ebc65163 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:14:11.638716 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc31d1f-59c9-4e60-84f6-9c99ebc65163 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:11.696023 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60187660-9768-442c-9d4f-48ae0014e9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:11.696926 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60187660-9768-442c-9d4f-48ae0014e9e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:11.697206 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60187660-9768-442c-9d4f-48ae0014e9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:11.697476 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60187660-9768-442c-9d4f-48ae0014e9e8 admin admin] PATH_INFO: `/v3/users/947fe2053da04a749e49712ac7b87d66` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:11.699255 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60187660-9768-442c-9d4f-48ae0014e9e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=947fe2053da04a749e49712ac7b87d66)` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:11.701263 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60187660-9768-442c-9d4f-48ae0014e9e8 admin admin] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:11.751096 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-60187660-9768-442c-9d4f-48ae0014e9e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '947fe2053da04a749e49712ac7b87d66'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:11.758390 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-60187660-9768-442c-9d4f-48ae0014e9e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '947fe2053da04a749e49712ac7b87d66'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:11.766324 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-60187660-9768-442c-9d4f-48ae0014e9e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '947fe2053da04a749e49712ac7b87d66'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:11.766989 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-60187660-9768-442c-9d4f-48ae0014e9e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '947fe2053da04a749e49712ac7b87d66'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:11.767590 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.notifications [None req-60187660-9768-442c-9d4f-48ae0014e9e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '947fe2053da04a749e49712ac7b87d66'} {{(pid=79888) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:11.785084 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1456/2894] 10.222.0.23 () {58 vars in 1223 bytes} [Fri May 6 15:14:11 2022] DELETE /identity/v3/users/947fe2053da04a749e49712ac7b87d66 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:14:12.054706 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627dac2-c606-4e71-84bc-d600faa70098 admin admin] Authenticating user token {{(pid=79887) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:14:12.056111 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.fernet_utils [None req-d627dac2-c606-4e71-84bc-d600faa70098 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79887) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:14:12.069576 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627dac2-c606-4e71-84bc-d600faa70098 admin admin] Validating token access rules against request {{(pid=79887) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:14:12.129025 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea31dd7-2355-4beb-bba4-d74f6a1ca04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66f1687219494ee0a402084110e075fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccc1ae27e989468199e57e6588faed16', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79887) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:14:12.129894 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea31dd7-2355-4beb-bba4-d74f6a1ca04f admin admin] REQUEST_METHOD: `DELETE` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:14:12.130181 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea31dd7-2355-4beb-bba4-d74f6a1ca04f admin admin] SCRIPT_NAME: `/identity` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:14:12.130460 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea31dd7-2355-4beb-bba4-d74f6a1ca04f admin admin] PATH_INFO: `/v3/projects/b3729a0fe92f482481bcf0b4cff9cb1e` {{(pid=79887) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:14:12.131635 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea31dd7-2355-4beb-bba4-d74f6a1ca04f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3729a0fe92f482481bcf0b4cff9cb1e)` {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:14:12.133585 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea31dd7-2355-4beb-bba4-d74f6a1ca04f admin admin] RBAC: Authorization granted {{(pid=79887) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:14:12.183180 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-1ea31dd7-2355-4beb-bba4-d74f6a1ca04f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3729a0fe92f482481bcf0b4cff9cb1e', 'request_id': 'req-1ea31dd7-2355-4beb-bba4-d74f6a1ca04f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:12.188625 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-1ea31dd7-2355-4beb-bba4-d74f6a1ca04f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3729a0fe92f482481bcf0b4cff9cb1e', 'request_id': 'req-1ea31dd7-2355-4beb-bba4-d74f6a1ca04f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:12.196033 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-1ea31dd7-2355-4beb-bba4-d74f6a1ca04f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3729a0fe92f482481bcf0b4cff9cb1e', 'request_id': 'req-1ea31dd7-2355-4beb-bba4-d74f6a1ca04f'} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:12.208131 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-1ea31dd7-2355-4beb-bba4-d74f6a1ca04f admin admin] The token cache is being invalidate because project b3729a0fe92f482481bcf0b4cff9cb1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79887) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 06 15:14:12.208425 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-1ea31dd7-2355-4beb-bba4-d74f6a1ca04f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:12.209121 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: DEBUG keystone.notifications [None req-1ea31dd7-2355-4beb-bba4-d74f6a1ca04f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79887) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 06 15:14:12.210718 dv-cinder-840678-2-iscsi devstack@keystone.service[79887]: [pid: 79887|app: 0|req: 1439/2895] 10.222.0.23 () {58 vars in 1232 bytes} [Fri May 6 15:14:12 2022] DELETE /identity/v3/projects/b3729a0fe92f482481bcf0b4cff9cb1e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 06 15:16:41.441541 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60187660-9768-442c-9d4f-48ae0014e9e8 admin admin] Authenticating user token {{(pid=79888) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 06 15:16:41.443636 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-60187660-9768-442c-9d4f-48ae0014e9e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:16:41.523549 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60187660-9768-442c-9d4f-48ae0014e9e8 admin admin] Validating token access rules against request {{(pid=79888) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 06 15:16:41.590970 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf1093b-0e79-41aa-a0b4-b733f46e90d8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887a619ef5b5417db98ff15a9f712724', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5398feee6eb490cb49bb261448a1a9b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79888) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 06 15:16:41.591709 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1093b-0e79-41aa-a0b4-b733f46e90d8 service placement] REQUEST_METHOD: `GET` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 06 15:16:41.592131 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1093b-0e79-41aa-a0b4-b733f46e90d8 service placement] SCRIPT_NAME: `/identity` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 06 15:16:41.592451 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1093b-0e79-41aa-a0b4-b733f46e90d8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=79888) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 06 15:16:41.594427 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.fernet_utils [None req-cdf1093b-0e79-41aa-a0b4-b733f46e90d8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79888) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 06 15:16:41.608561 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf1093b-0e79-41aa-a0b4-b733f46e90d8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 06 15:16:41.611626 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf1093b-0e79-41aa-a0b4-b733f46e90d8 service placement] RBAC: Authorization granted {{(pid=79888) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 06 15:16:41.632866 dv-cinder-840678-2-iscsi devstack@keystone.service[79888]: [pid: 79888|app: 0|req: 1457/2896] 10.222.0.23 () {60 vars in 1365 bytes} [Fri May 6 15:16:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)